A股上市公司传智教育(股票代码 003032)旗下技术交流社区北京昌平校区

 找回密码
 加入黑马

QQ登录

只需一步,快速开始

2021 年 6 月 15 日,中央网信办宣布在全国范围内开展为期 2 个月的 " 清朗 · ‘饭圈’乱象整治 " 专项行动,紧接着吴亦凡事件彻底点燃全网风暴。2021-08-27 中央网信办下发通知要求取消明星艺人榜单,严禁呈现互撕信息。
素有 " 饭圈大本营 " 之称的微博,则下架了 2018 年引进的 " 明星超话积分助力机制 ",规则修改后的明星超话将根据活跃度展示,包含在超话内签到、互动等综合维度,排名依然存在。如何才能及时的在微博明星超话进行签到和排名的呢?下面的的代码运行就可以获取相应的信息,可以试试
[Java] 纯文本查看 复制代码
package htmlunit;

import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;

import com.gargoylesoftware.htmlunit.BrowserVersion;
import com.gargoylesoftware.htmlunit.NicelyResynchronizingAjaxController;
import com.gargoylesoftware.htmlunit.WebClient;
import com.gargoylesoftware.htmlunit.html.HtmlPage;

public class HtmlunitDemo {
    // 代理服务器(产品官网 www.16yun.cn)
    final static String proxyHost = "t.16yun.cn";
    final static Integer proxyPort = 31111;

    // 代理验证信息
    final static String proxyUser = "USERNAME";
    final static String proxyPass = "PASSWORD";

    public static void main(String[] args) {

        CredentialsProvider credsProvider = new BasicCredentialsProvider();
        credsProvider.setCredentials(

        new AuthScope(proxyHost, proxyPort),
        new UsernamePasswordCredentials(proxyUser, proxyPass));


        WebClient webClient = new WebClient(BrowserVersion.CHROME,proxyHost, proxyPort);


        webClient.setCredentialsProvider(credsProvider);


        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
        webClient.getOptions().setActiveXNative(false);
        webClient.getOptions().setCssEnabled(false);

        HtmlPage page = null;

        try {
            page = webClient.getPage("https://weibo.com/p/100808fb050d6c9bcc97ed6836bb9ab9b3da7f/super_index");
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            webClient.close();
        }

        webClient.waitForBackgroundJavaScript(30000);


        String pageXml = page.asXml();

        System.out.println(pageXml);
    }
}

0 个回复

您需要登录后才可以回帖 登录 | 加入黑马