快捷導(dǎo)航

Python selenium打開瀏覽器指定端口實現(xiàn)接續(xù)操作

更新時間：2025年05月16日 09:52:28 作者：Looooking

這篇文章主要為大家詳細介紹了Python selenium如何實現(xiàn)打開瀏覽器指定端口后進行接續(xù)操作,文中的示例代碼講解詳細,有需要的小伙伴可以了解下

一般使用 selenium 進行數(shù)據(jù)爬取時，常用處理流程是讓 selenium 從打開瀏覽器開始，完成全流程的所有操作。但是有時候，我們希望用戶先自己打開瀏覽器進入指定網(wǎng)頁，完成登錄認證等一系列操作之后（比如用戶、密碼、短信驗證碼及各種難處理的圖形驗證碼之類），再讓 selenium 從登錄后的頁面進行接續(xù)操作爬取數(shù)據(jù)。那么如何才能將前后操作接續(xù)起來呢？

常規(guī)操作

常規(guī)操作一般使用下面的這種方式，設(shè)置初始參數(shù)后直接使用 get 方法去打開網(wǎng)頁。

from selenium import webdriver
 
 
class DriverClass:
    def __init__(self):
        self.driver = self._init_driver()
 
    def _init_driver(self):
        try:
            option = webdriver.ChromeOptions()
            option.add_experimental_option('excludeSwitches', ['enable-automation'])
            option.add_experimental_option('useAutomationExtension', False)
            prefs = dict()
            prefs['credentials_enable_service'] = False
            prefs['profile.password_manager_enable'] = False
            prefs['profile.name'] = "Person 1"
            option.add_experimental_option('prefs', prefs)
            option.add_argument('--disable-gpu')
            option.add_argument("--disable-blink-features=AutomationControlled")
            option.add_argument('--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"')
            option.add_argument('--no-sandbox')
            option.add_argument('ignore-certificate-errors')
            driver = webdriver.Chrome(r"./driver/chromedriver.exe", options=option)
            driver.implicitly_wait(2)
            driver.maximize_window()
            return driver
        except Exception as e:
            raise e
 
    def get_driver(self) -> webdriver.Chrome:
        if isinstance(self.driver, webdriver.Chrome):
            return self.driver
        raise Exception('初始化瀏覽器失敗')
 
 
if __name__ == '__main__':
    dc = DriverClass()
    driver = dc.get_driver()
    print(driver)
    driver.get("https://www.baidu.com")

接續(xù)操作

接續(xù)操作主要通過在打開瀏覽器時，都設(shè)置相同的接口來完成前后的銜接（不然 selenium 不知道要從哪個瀏覽器頁面進行接續(xù)）。

用戶打開瀏覽器

用戶手動打開瀏覽器時，指定對應(yīng)的端口（這里設(shè)置的是 9527）及數(shù)據(jù)目錄（自己自定義自定一個）。

C:\Program Files\Google\Chrome\Application>chrome.exe --remote-debugging-port=9527 --user-data-dir="E:\lky_project\tmp_project\handle_qcc_data\\chrome_user_data"

執(zhí)行完上面的命令以后，會打開一個新的瀏覽器頁面。

打開瀏覽器后，用戶可以手動輸入相應(yīng)頁面，完成相應(yīng)的用戶登錄認證等操作。

程序接續(xù)瀏覽器

selenium 通過增加下面的配置參數(shù)

option.add_experimental_option("debuggerAddress", "127.0.0.1:9527")

來打開并接續(xù)處理用戶已經(jīng)打開的指定端口的瀏覽器。之后，程序就可以通過瀏覽器句柄去接續(xù)處理后續(xù)的任務(wù)了。

driver_class.py

from selenium import webdriver
 
 
class DriverClass:
    def __init__(self):
        self.driver = self._init_driver()
 
    def _init_driver(self):
        try:
            option = webdriver.ChromeOptions()
            # option.add_experimental_option('excludeSwitches', ['enable-automation'])
            # option.add_experimental_option('useAutomationExtension', False)
            # prefs = dict()
            # prefs['credentials_enable_service'] = False
            # prefs['profile.password_manager_enable'] = False
            # prefs['profile.name'] = "Person 1"
            # option.add_experimental_option('prefs', prefs)
            option.add_argument('--disable-gpu')
            option.add_argument("--disable-blink-features=AutomationControlled")
            option.add_argument('--user-agent="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"')
            option.add_argument('--no-sandbox')
            option.add_argument('ignore-certificate-errors')
            option.add_experimental_option("debuggerAddress", "127.0.0.1:9527")
            driver = webdriver.Chrome(r"./driver/chromedriver.exe", options=option)
            driver.implicitly_wait(2)
            # driver.maximize_window()
            return driver
        except Exception as e:
            raise e
 
    def get_driver(self) -> webdriver.Chrome:
        if isinstance(self.driver, webdriver.Chrome):
            return self.driver
        raise Exception('初始化瀏覽器失敗')
 
 
if __name__ == '__main__':
    dc = DriverClass()
    driver = dc.get_driver()
    print(driver)
    # 程序使用接續(xù)后的瀏覽器句柄 driver 完成后續(xù)操作

注意事項

注意看，我上面的接續(xù)操作函數(shù)，有一部分的參數(shù)設(shè)置是注釋掉的。這是因為接續(xù)是從已經(jīng)打開的瀏覽器接收繼續(xù)進行操作，有部分的參數(shù)在用戶打開瀏覽器的時候就已經(jīng)設(shè)定了，所以不再支持通過接續(xù)的方式繼續(xù)重復(fù)設(shè)置。

實戰(zhàn)示例

比如在手動打開指定 9527 端口的瀏覽器后，登錄企查查進入高級搜索，然后使用程序獲取具有相應(yīng)資質(zhì)的企業(yè)數(shù)目（操作太頻繁可能觸發(fā)校驗或封號，請謹慎操作！），最后生成結(jié)果文件 data.json（中途可能會異常中斷，可以做成下面這種利用 data.json 實現(xiàn)的斷點續(xù)查的方式，這樣，后續(xù)再次運行也只會查詢未查詢過的資質(zhì)數(shù)據(jù)）。

driver_class.py 用上面的就可以。

main.py

import json
import re
import time
 
from selenium.webdriver.common.by import By
from driver_class import DriverClass
 
dc = DriverClass()
driver = dc.get_driver()
xpath_prefix = '//div/div/div/div/span[text()="資質(zhì)證書"]/following-sibling::div'
 
 
def checkbox_select(element_checkbox):
    """復(fù)選框選中"""
    class_attribute = element_checkbox.get_attribute("class")
    if "checked" not in class_attribute:
        element_checkbox.find_element(By.XPATH, './span[@class="qccd-tree-checkbox-inner"]').click()
 
 
def checkbox_unselect(element_checkbox):
    """復(fù)選框取消選中"""
    class_attribute = element_checkbox.get_attribute("class")
    if "checked" in class_attribute:
        element_checkbox.find_element(By.XPATH, './span[@class="qccd-tree-checkbox-inner"]').click()
 
 
def get_amount(element_checkbox):
    """獲取對應(yīng)復(fù)選框?qū)?yīng)的企業(yè)數(shù)目"""
    checkbox_select(element_checkbox)
    xpath_confirm = xpath_prefix + '/div/div/div/div/div[text()="確定"]'
    driver.find_element(By.XPATH, xpath_confirm).click()
    time.sleep(0.5)
    try:
        xpath_result = '//div/div/div[@class="search-btn limit-svip"]'
        result = str(driver.find_element(By.XPATH, xpath_result).text)
    except Exception as e:
        print(f"異常: {str(e)}")
        result = "0"
    result = result.replace(",", "")
    match_object = re.search("(\d+)", result)
    amount = match_object.group(1)
    print(f"數(shù)目：{amount}")
    # 清除結(jié)果，避免點擊選擇項時誤點擊關(guān)閉
    xpath_clear = '//div/div/a[contains(text(), "清除")]'
    try:
        driver.find_element(By.XPATH, xpath_clear).click()
    except:
        pass
    xpath_select = xpath_prefix + '[@class="trigger-container"]'
    driver.find_element(By.XPATH, xpath_select).click()
    time.sleep(0.2)
    checkbox_unselect(element_checkbox)
    return amount
 
 
def extend_options():
    """展開折疊項并獲取數(shù)據(jù)，只展開三層"""
    # json.dump(data, open("data.json", 'w', encoding="utf-8"), indent=2, ensure_ascii=False)
    try:
        data = json.load(open("data.json", encoding="utf-8"))
    except:
        data = {}
    try:
        xpath_first_class = xpath_prefix + '//div/ul/li[@role="treeitem"]'
        # xpath_first_class = xpath_prefix + '//div/ul/li/span[contains(@class, "qccd-tree-switcher")]'
        first_item_list = driver.find_elements(By.XPATH, xpath_first_class)
        for item_li in first_item_list:
            text_dk1 = item_li.find_element(By.XPATH, './span/span/div/span[@class="text-dk"]').text
            data[text_dk1] = data.get(text_dk1, {})
            print(f"{text_dk1}")
            switcher = item_li.find_element(By.XPATH, './span[contains(@class, "qccd-tree-switcher")]')
            class_attribute = switcher.get_attribute("class")
            element_checkbox = item_li.find_element(By.XPATH, './span[contains(@class, "checkbox")]')
            if "close" in class_attribute:
                switcher.click()
                time.sleep(0.1)
            elif "noop" in class_attribute:
                # 當(dāng)前節(jié)點沒有子節(jié)點
                if not data[text_dk1]:
                    amount = get_amount(element_checkbox)
                    data[text_dk1] = amount
                continue
            # 點開以后，下層級的 ul/li 會展示出來
            second_item_list = item_li.find_elements(By.XPATH, "./ul/li")
            for second_item_li in second_item_list:
                text_dk2 = second_item_li.find_element(By.XPATH, './span/span/div/span[@class="text-dk"]').text
                data[text_dk1][text_dk2] = data[text_dk1].get(text_dk2, {})
                print(f"--{text_dk2}")
                switcher = second_item_li.find_element(By.XPATH, './span[contains(@class, "qccd-tree-switcher")]')
                class_attribute = switcher.get_attribute("class")
                element_checkbox = second_item_li.find_element(By.XPATH, './span[contains(@class, "checkbox")]')
                if "close" in class_attribute:
                    switcher.click()
                    time.sleep(0.1)
                elif "noop" in class_attribute:
                    # 當(dāng)前節(jié)點沒有子節(jié)點
                    if not data[text_dk1][text_dk2]:
                        amount = get_amount(element_checkbox)
                        data[text_dk1][text_dk2] = amount
                    continue
                # 點開以后，下層級的 ul/li 會展示出來
                third_item_list = second_item_li.find_elements(By.XPATH, "./ul/li")
                for third_item_li in third_item_list:
                    text_dk3 = third_item_li.find_element(By.XPATH, './span/span/div/span[@class="text-dk"]').text
                    data[text_dk1][text_dk2][text_dk3] = data[text_dk1][text_dk2].get(text_dk3, {})
                    print(f"----{text_dk3}")
                    switcher = third_item_li.find_element(By.XPATH, './span[contains(@class, "qccd-tree-switcher")]')
                    class_attribute = switcher.get_attribute("class")
                    # 到第三層時，不再展開，直接選擇復(fù)選框
                    element_checkbox = third_item_li.find_element(By.XPATH, './span[contains(@class, "checkbox")]')
                    if not data[text_dk1][text_dk2][text_dk3]:
                        amount = get_amount(element_checkbox)
                        data[text_dk1][text_dk2][text_dk3] = amount
    except Exception as e:
        raise e
    finally:
        json.dump(data, open("data.json", 'w', encoding="utf-8"), indent=2, ensure_ascii=False)
 
 
def spider_data():
    # 嘗試關(guān)閉資質(zhì)證書選擇框、清除所選項
    xpath_close = xpath_prefix + '/div/div/div/a[@class="nclose"]'
    xpath_clear = '//div/div/a[contains(text(), "清除")]'
    try:
        driver.find_element(By.XPATH, xpath_close).click()
    except:
        pass
    try:
        driver.find_element(By.XPATH, xpath_clear).click()
    except:
        pass
    # 點擊資質(zhì)證書選擇框
    xpath_select = xpath_prefix + '[@class="trigger-container"]'
    driver.find_element(By.XPATH, xpath_select).click()
    time.sleep(2)
    extend_options()
    # 取消按鈕
    xpath_cancel = xpath_prefix + '/div/div/div/div/div[text()="取消"]'
    # 確定按鈕
    xpath_confirm = xpath_prefix + '/div/div/div/div/div[text()="確定"]'
    driver.find_element(By.XPATH, xpath_confirm).click()
 
 
if __name__ == '__main__':
    spider_data()

最后可以得到生成的 data.json 文件如下：

{
  "建筑業(yè)資質(zhì)": {
    "工程設(shè)計資質(zhì)證書": {
      "工程設(shè)計專項資質(zhì)": "26329",
      "建筑工程設(shè)計事務(wù)所": "356",
      "工程設(shè)計行業(yè)資質(zhì)": "4487",
      "工程設(shè)計專業(yè)資質(zhì)": "19902",
      "工程設(shè)計綜合資質(zhì)": "98"
    },
    "工程勘察資質(zhì)證書": {
      "工程勘察綜合資質(zhì)": "377",
      "工程勘察專業(yè)資質(zhì)": "7464",
      "工程勘察勞務(wù)資質(zhì)": "3019"
    },
...
  },
  "食品農(nóng)產(chǎn)品認證": {
    "有機產(chǎn)品(OGA)": "49868",
    "良好農(nóng)業(yè)規(guī)范(GAP)": "6449",
    "食品質(zhì)量認證(酒類)": "151",
    "綠色食品認證": "34723",
    "綠色市場認證": "318",
    "無公害農(nóng)產(chǎn)品": "31067",
    "食品安全管理體系認證": "72075",
    "危害分析與關(guān)鍵控制點認證": "51844",
    "乳制品生產(chǎn)企業(yè)良好生產(chǎn)規(guī)范認證": "445",
    "乳制品生產(chǎn)企業(yè)危害分析與關(guān)鍵控制點(HACCP)體系認證": "570",
    "飼料產(chǎn)品": "85"
  },
  "其他資質(zhì)": {
    "辦學(xué)許可證": "192010",
    "代理記賬許可證書": "34588",
    "會計師事務(wù)所執(zhí)業(yè)證書": "12252",
    "DOC證書": "982",
    "SMC證書": "1886",
    "名特優(yōu)新農(nóng)產(chǎn)品證書": "1818",
    "招投標類綜合資質(zhì)": "36317",
    "區(qū)塊鏈信息服務(wù)備案": "2765",
    "醫(yī)療機構(gòu)執(zhí)業(yè)許可證": "570877",
    "CCC工廠認證": "16154",
    "衛(wèi)生許可證": "3244"
  }
}

以上就是Python selenium打開瀏覽器指定端口實現(xiàn)接續(xù)操作的詳細內(nèi)容，更多關(guān)于Python selenium瀏覽器的資料請關(guān)注腳本之家其它相關(guān)文章！

您可能感興趣的文章: