Java爬蟲(chóng)實(shí)現(xiàn)Jsoup利用dom方法遍歷Document對(duì)象
更新時(shí)間：2022年05月17日 16:49:51 作者：Cyril_KI
本文主要介紹了Java爬蟲(chóng)實(shí)現(xiàn)Jsoup利用dom方法遍歷Document對(duì)象，文中通過(guò)示例代碼介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值，需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)學(xué)習(xí)吧
先給出網(wǎng)頁(yè)地址：
https://wall.alphacoders.com/featured.php?lang=Chinese
主要步驟：
利用Jsoup的connect方法獲取Document對(duì)象
String html = "https://wall.alphacoders.com/featured.php?lang=Chinese"; Document doc = Jsoup.connect(html).get();
內(nèi)容過(guò)長(zhǎng)，就不再顯示。
我們以這部分為例：
<ul class="nav nav-pills">
<li><a rel="external nofollow" rel="external nofollow" >About Us</a></li>
<li><a rel="external nofollow" rel="external nofollow" >FAQ</a></li>
<li><a rel="external nofollow" rel="external nofollow" >Privacy Policy</a></li>
<li><a rel="external nofollow" rel="external nofollow" >Terms Of Service</a></li>
<li><a rel="external nofollow" rel="external nofollow" >Acceptable Use</a></li>
<li><a rel="external nofollow" rel="external nofollow" >Etiquette</a></li>
<li><a rel="external nofollow" rel="external nofollow" >Advertise With Us</a></li>
<li><a id="change_consent">Change Consent</a></li>
</ul>
我們先找到所有的ul：
Elements elements = doc.getElementsByTag("ul");
輸出如下：
<ul class="nav navbar-nav center"> <li> <a title="Submit Wallpapers" rel="external nofollow" ><i class="el el-circle-arrow-up"></i> 提交</a> </li> <li> <a rel="external nofollow" ><i class="el el-gift"></i> 精美獎(jiǎng)品</a> </li> </ul> <ul class="nav navbar-nav navbar-right center"> <li> <a href="language.php?lang=Chinese" rel="external nofollow" > <img src="https://static.alphacoders.com/wa/Chinese-flag.png" alt="Chinese-flag"> 中文 </a> </li> <li> <a rel="external nofollow" ><i class="el el-user"></i> 登錄</a> </li> <li> <a rel="external nofollow" ><i class="el el-edit"></i> 注冊(cè)</a> </li> </ul> <ul class="pagination"> <li class="active"><a id="prev_page" href="#" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >< 上一頁(yè)</a></li> <li class="active"><a>1</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >2</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >3</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >4</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >5</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >6</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >7</a></li> <li><a rel="external nofollow" >8</a></li> <li><a rel="external nofollow" >9</a></li> <li><a rel="external nofollow" >10</a></li> <li><a>...</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >319</a></li> <li><a id="next_page" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >下一頁(yè) ></a></li> </ul> <ul class="pagination"> <li class="active"><a href="#" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >< 上一頁(yè)</a></li> <li class="active"><a>1</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >2</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >3</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >4</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >5</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >6</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >7</a></li> <li><a>...</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >319</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >下一頁(yè) ></a></li> </ul> <ul class="pagination"> <li class="active"><a href="#" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" ><< </a></li> <li class="active"><a href="#" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >< 上一頁(yè)</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >下一頁(yè) ></a></li> <li><a title="末頁(yè) (319)" rel="external nofollow" rel="external nofollow" rel="external nofollow" > >></a></li> </ul> <ul class="pagination"> <li class="active"><a href="#" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >1</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" rel="external nofollow" >2</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >3</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >4</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >5</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >6</a></li> <li><a rel="external nofollow" rel="external nofollow" rel="external nofollow" >7</a></li> </ul> <ul class="nav nav-pills"> <li><a rel="external nofollow" rel="external nofollow" >About Us</a></li> <li><a rel="external nofollow" rel="external nofollow" >FAQ</a></li> <li><a rel="external nofollow" rel="external nofollow" >Privacy Policy</a></li> <li><a rel="external nofollow" rel="external nofollow" >Terms Of Service</a></li> <li><a rel="external nofollow" rel="external nofollow" >Acceptable Use</a></li> <li><a rel="external nofollow" rel="external nofollow" >Etiquette</a></li> <li><a rel="external nofollow" rel="external nofollow" >Advertise With Us</a></li> <li><a id="change_consent">Change Consent</a></li> </ul>
可以發(fā)現(xiàn)class為"nav nav-pills"的只有一個(gè)，我們找到它：
Elements elements = doc.getElementsByTag("ul");
//System.out.println(elements);
Element tempElement = null;
for(Element element : elements) {
if (element.className().equals("nav nav-pills")) {
tempElement = element;
//System.out.println(element.className());
break;
}
}
循環(huán)遍歷這個(gè)ul，輸出其中每一個(gè)li里每一個(gè)a的href和rel屬性：
Elements li = tempElement.getElementsByTag("li");
for(Element element : li) {
Elements element2 = element.getElementsByTag("a");
for(Element element3 : element2) {
String hrefString = element3.attr("href");
String relString = element3.attr("rel");
if(hrefString != "" && relString != "") {
System.out.println("href=" + hrefString + " rel="external nofollow" rel="external nofollow" " + "rel=" + relString);
}
}
}
最終結(jié)果：
href=https://alphacoders.com/site/about-us rel=nofollow
href=https://alphacoders.com/site/faq rel=nofollow
href=https://alphacoders.com/site/privacy rel=nofollow
href=https://alphacoders.com/site/tos rel=nofollow
href=https://alphacoders.com/site/acceptable_use rel=nofollow
href=https://alphacoders.com/site/etiquette rel=nofollow
href=https://alphacoders.com/site/advertising rel=nofollow
完整代碼：
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
import java.io.IOException;
import org.jsoup.Jsoup;
/**?
?* @ClassName: Jsoup_Test
?* @description:?
?* @author: KI
?* @Date: 2020年8月17日 下午8:15:14
?*/
public class Jsoup_Test {
?? ?public static void main(String[] args) throws IOException {
?? ??? ?// TODO 自動(dòng)生成的方法存根
?? ??? ?
?? ??? ?String html = "https://wall.alphacoders.com/featured.php?lang=Chinese";
?? ??? ?Document doc = Jsoup.connect(html).get();
?? ??? ?
?? ??? ?System.out.println(doc);
?? ??? ?Elements elements = doc.getElementsByTag("ul");
?? ??? ?//System.out.println(elements);
?? ??? ?Element tempElement = null;
?? ??? ?for(Element element : elements) {
?? ??? ??? ?if (element.className().equals("nav nav-pills")) {
?? ??? ??? ??? ?tempElement = element;
?? ??? ??? ??? ?//System.out.println(element.className());
?? ??? ??? ??? ?break;
?? ??? ??? ?}
?? ??? ?}
?? ??? ?System.out.println(tempElement);
?? ??? ?Elements li = tempElement.getElementsByTag("li");
?? ??? ?for(Element element : li) {
?? ??? ??? ?Elements element2 = element.getElementsByTag("a");
?? ??? ??? ?for(Element element3 : element2) {
?? ??? ??? ??? ?String hrefString = element3.attr("href");
?? ??? ??? ??? ?String relString = element3.attr("rel");
?? ??? ??? ??? ?if(hrefString != "" && relString != "") {
?? ??? ??? ??? ??? ?System.out.println("href=" + hrefString + " rel="external nofollow" rel="external nofollow" " + "rel=" + relString);
?? ??? ??? ??? ?}
?? ??? ??? ?}
?? ??? ?}?? ??? ?
?? ?}
}到此這篇關(guān)于Java爬蟲(chóng)實(shí)現(xiàn)Jsoup利用dom方法遍歷Document對(duì)象的文章就介紹到這了,更多相關(guān)Java Jsoup遍歷Document對(duì)象內(nèi)容請(qǐng)搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關(guān)文章希望大家以后多多支持腳本之家！
相關(guān)文章
這篇文章主要介紹了Java基礎(chǔ)教程之接口的繼承與抽象類,本文介紹了接口繼承、接口的多重繼承以及抽象類的知識(shí),需要的朋友可以參考下2014-09-09
mybatis-generator-gui根據(jù)需求改動(dòng)示例
這篇文章主要為大家介紹了mybatis-generator-gui根據(jù)需求改動(dòng)示例,有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪2023-09-09
這篇文章主要介紹了Java中精確的浮點(diǎn)運(yùn)算操作方法,結(jié)合具體實(shí)例形式分析了java浮點(diǎn)數(shù)運(yùn)算的相關(guān)函數(shù)、使用技巧與注意事項(xiàng),需要的朋友可以參考下2017-06-06
JSON序列化導(dǎo)致Long類型被搞成Integer的坑及解決
這篇文章主要介紹了JSON序列化導(dǎo)致Long類型被搞成Integer的坑及解決方案，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教2022-01-01
如何使用stream從List對(duì)象中獲取某列數(shù)據(jù)
這篇文章主要介紹了如何使用stream從List對(duì)象中獲取某列數(shù)據(jù)問(wèn)題，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教2022-12-12
idea下載svn的項(xiàng)目并且運(yùn)行操作
這篇文章主要介紹了idea下載svn的項(xiàng)目并且運(yùn)行操作，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。一起跟隨小編過(guò)來(lái)看看吧2020-09-09
這篇文章主要介紹了IDEA?中使用?Hudi的相關(guān)知識(shí)，本文通過(guò)實(shí)例代碼給大家介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或工作具有一定的參考借鑒價(jià)值，需要的朋友可以參考下2022-06-06
Java使用Maven BOM統(tǒng)一管理版本號(hào)的實(shí)現(xiàn)
這篇文章主要介紹了Java使用Maven BOM統(tǒng)一管理版本號(hào)的實(shí)現(xiàn)，文中通過(guò)示例代碼介紹的非常詳細(xì)，對(duì)大家的學(xué)習(xí)或者工作具有一定的參考學(xué)習(xí)價(jià)值，需要的朋友們下面隨著小編來(lái)一起學(xué)習(xí)學(xué)習(xí)吧2021-04-04
springboot 如何取消starter的自動(dòng)注入
這篇文章主要介紹了springboot 如何取消starter的自動(dòng)注入操作，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教2021-09-09
Springboot項(xiàng)目與vue項(xiàng)目整合打包的實(shí)現(xiàn)方式
這篇文章主要介紹了Springboot項(xiàng)目與vue項(xiàng)目整合打包的實(shí)現(xiàn)方式,本文通過(guò)兩種方式給大家介紹的非常詳細(xì)，具有一定的參考借鑒價(jià)值，需要的朋友可以參考下2019-07-07