最新国产好看的视频,伊人天堂AV在线,国产Aaaaaa视频,蜜臀视频在线观看一区,人妻av色图,密臀久久久精品影片,青青视频免费观看毛片,久草在线观看视,国产三级精品色情在线

Python實(shí)現(xiàn)批量將CSV按行或按列拆分

 更新時(shí)間:2026年03月04日 08:27:20   作者:小莊-Python辦公  
這篇文章主要介紹了一個(gè)基于Python標(biāo)準(zhǔn)庫(kù)csv和PyQt5的高性能CSV拆分工具,文中的示例代碼講解詳細(xì),感興趣的小伙伴可以跟隨小編一起學(xué)習(xí)一下

本文完整拆解一個(gè)可用于生產(chǎn)的 CSV 拆分工具,從架構(gòu)、核心算法、GUI 與多線程到異常與性能邊界,全部基于 Python 標(biāo)準(zhǔn)庫(kù) csv 與 PyQt5 實(shí)現(xiàn)。

效果圖

需求與目標(biāo)

核心目標(biāo)是實(shí)現(xiàn)一個(gè)桌面 GUI:

  • 多文件選擇與拖拽列表
  • 按行拆分與按列拆分兩種模式,參數(shù)聯(lián)動(dòng)
  • 可配置輸出目錄、編碼、分隔符、命名規(guī)則與覆蓋策略
  • 支持 1GB / 500 萬(wàn)行規(guī)模,內(nèi)存占用低,GUI 不阻塞
  • 實(shí)時(shí)進(jìn)度、日志與異常提示,支持取消與清理

總體設(shè)計(jì)

項(xiàng)目結(jié)構(gòu)極簡(jiǎn),僅一個(gè)入口腳本:

  • GUI 與業(yè)務(wù)邏輯全部在 [main.py]
  • 核心拆分函數(shù) split_csv_file 完成流式拆分與異常捕獲
  • Worker 在 QThread 中執(zhí)行,保證 GUI 響應(yīng)
  • 日志、進(jìn)度通過(guò) Qt 信號(hào)回傳 UI

設(shè)計(jì)重點(diǎn)是把“耗時(shí) IO 與 CPU”放在后臺(tái)線程,把“交互與展示”留給主線程。

CSV 拆分核心邏輯

核心函數(shù)是 [split_csv_file],僅使用 csv.reader/csv.writer,滿足低內(nèi)存與高兼容性。

1. 統(tǒng)一設(shè)置與結(jié)果模型

通過(guò) SplitSettingsFileResult 描述輸入配置與輸出狀態(tài):

  • 選擇拆分模式、編碼、分隔符
  • 命名規(guī)則(前綴/后綴/序號(hào)格式)
  • 是否保留表頭、覆蓋策略
  • 統(tǒng)一錯(cuò)誤信息與 traceback

2. 按行拆分

按行拆分包含兩種子模式:

  • 按固定行數(shù)切分
  • 按唯一列值分組輸出

按固定行數(shù)

關(guān)鍵點(diǎn):

  • 每達(dá)到 rows_per_file 就關(guān)閉當(dāng)前輸出文件,啟動(dòng)下一個(gè)
  • 表頭按需寫(xiě)入
  • 通過(guò) row_index 控制進(jìn)度刷新,避免頻繁更新 UI

示例片段(僅為理解):

if count == 0:
    seq_text = format_sequence(seq, settings.seq_format)
    filename = f"{settings.prefix}{seq_text}{settings.suffix}.csv"
    out_file = os.path.join(settings.output_dir, filename)

按唯一列值

核心策略是使用 WriterCache 復(fù)用輸出文件句柄,避免每行開(kāi)關(guān)文件:

  • WriterCache 維護(hù)有限數(shù)量打開(kāi)文件
  • 超出上限后按 LRU 關(guān)閉句柄
  • 已創(chuàng)建文件用 append 追加,避免重復(fù)寫(xiě)表頭

3. 按列拆分與二次按行

按列拆分本質(zhì)上也是“按唯一值分組輸出”,并可選二次按行拆分:

  • 先按列值生成文件名
  • 若開(kāi)啟二次拆分,按行數(shù)再切分出 value_序號(hào).csv

4. 文件名安全與序號(hào)格式

為防止非法字符,統(tǒng)一使用 sanitize_filename 處理列值:

  • 替換系統(tǒng)禁止字符
  • 空值統(tǒng)一成 EMPTY
  • 過(guò)長(zhǎng)截?cái)?/li>

序號(hào)格式支持兩種:

  • 純數(shù)字模板,如 0001
  • format 模式,如 {0:04d}

GUI 設(shè)計(jì)與交互流

GUI 核心在拆分成多個(gè)可復(fù)用面板。

1. 文件選擇與拖拽

FileListWidget 繼承 QListWidget,實(shí)現(xiàn)拖拽導(dǎo)入:

  • 過(guò)濾 csv 后綴
  • 自動(dòng)去重

2. 模式切換

按行/按列通過(guò) QStackedWidget 互斥顯示:

  • self.mode_rowself.mode_col 綁定
  • _switch_mode 切換索引

3. 參數(shù)聯(lián)動(dòng)與列名讀取

當(dāng)文件列表變化或編碼/分隔符變化時(shí),自動(dòng)刷新列名:

  • 讀取首行作為表頭
  • 更新“按行唯一列”和“按列拆分列”下拉框

4. 輸出配置與命名規(guī)則

統(tǒng)一面板提供:

  • 輸出目錄選擇
  • 分隔符(逗號(hào)/分號(hào)/制表符/豎線)
  • 編碼(UTF-8/UTF-8-sig/GBK)
  • 覆蓋或跳過(guò)策略
  • 命名規(guī)則(前綴/后綴/序號(hào)格式)

多線程與任務(wù)生命周期

耗時(shí)任務(wù)在 QThread 中執(zhí)行:

  • run() 循環(huán)處理文件
  • 向 UI 發(fā)射總體進(jìn)度與單文件進(jìn)度
  • 每個(gè)文件獨(dú)立成功/失敗計(jì)數(shù),互不阻塞

調(diào)用流程:

  • _start() 創(chuàng)建線程與 Worker
  • 綁定信號(hào):log、progress、finished
  • worker_thread.start() 開(kāi)始后臺(tái)執(zhí)行

進(jìn)度與日志設(shè)計(jì)

進(jìn)度條

進(jìn)度分兩層:

  • 當(dāng)前文件進(jìn)度:讀取 file.tell() 與文件大小比例
  • 總體進(jìn)度:按文件大小加權(quán)

日志

日志輸出包含時(shí)間戳與異常堆棧:

  • timestamp() 統(tǒng)一格式
  • 失敗時(shí)記錄 traceback
  • 支持導(dǎo)出為文本文件

異常處理與容錯(cuò)策略

異常處理貫穿拆分流程:

  • 編碼錯(cuò)誤 UnicodeDecodeError
  • 列名缺失 ValueError
  • 磁盤(pán)空間不足 OSError + errno.ENOSPC

設(shè)計(jì)要點(diǎn):

  • 單文件失敗不影響其他文件
  • 失敗信息寫(xiě)入日志并彈窗提示
  • 失敗數(shù)在任務(wù)結(jié)束匯總

性能與內(nèi)存控制

針對(duì) 1GB 大文件設(shè)計(jì)要點(diǎn):

  • csv.reader 流式讀取,行級(jí)處理
  • 輸出文件使用 WriterCache 限制句柄數(shù)量
  • 不緩存整文件或整列內(nèi)容

內(nèi)存占用主要來(lái)自:

  • 現(xiàn)有行字符串對(duì)象
  • 已打開(kāi)的輸出句柄
  • GUI 層的日志與列表

對(duì)于“按唯一列值”的模式,如果唯一值極多:

  • 可適當(dāng)降低 max_open_files
  • 采用“跳過(guò)已存在文件”減少 IO

源代碼

'''
作者:小莊-Python辦公
公眾號(hào):小莊-Python學(xué)習(xí)
'''
import sys
import os
import csv
import traceback
import errno
from dataclasses import dataclass, field
from datetime import datetime
from typing import Callable, Dict, List, Optional, Tuple

from PyQt5.QtCore import Qt, QObject, pyqtSignal, QThread
from PyQt5.QtGui import QFont
from PyQt5.QtWidgets import (
    QApplication,
    QMainWindow,
    QWidget,
    QFileDialog,
    QListWidget,
    QListWidgetItem,
    QVBoxLayout,
    QHBoxLayout,
    QLabel,
    QPushButton,
    QRadioButton,
    QButtonGroup,
    QStackedWidget,
    QGroupBox,
    QLineEdit,
    QSpinBox,
    QComboBox,
    QCheckBox,
    QTextEdit,
    QMessageBox,
    QProgressBar,
)


def timestamp() -> str:
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


def sanitize_filename(value: str) -> str:
    if value is None:
        value = ""
    value = str(value).strip()
    if not value:
        value = "EMPTY"
    for ch in ['\\', '/', ':', '*', '?', '"', '<', '>', '|']:
        value = value.replace(ch, "_")
    value = value.replace("\n", "_").replace("\r", "_").replace("\t", "_")
    value = value.strip(" .")
    if not value:
        value = "EMPTY"
    if len(value) > 120:
        value = value[:120]
    return value


def format_sequence(seq: int, pattern: str) -> str:
    if "{" in pattern and "}" in pattern:
        return pattern.format(seq)
    width = max(1, len(pattern))
    return str(seq).zfill(width)


def ensure_dir(path: str) -> None:
    if not os.path.isdir(path):
        os.makedirs(path, exist_ok=True)


@dataclass
class SplitSettings:
    mode: str
    row_mode: str
    rows_per_file: int
    unique_column: str
    col_column: str
    col_secondary_split: bool
    col_secondary_rows: int
    prefix: str
    suffix: str
    seq_format: str
    keep_header: bool
    delimiter: str
    encoding: str
    output_dir: str
    on_exists: str
    max_open_files: int = 64


@dataclass
class FileResult:
    file_path: str
    success: bool
    created_files: List[str] = field(default_factory=list)
    skipped_files: List[str] = field(default_factory=list)
    error_message: str = ""
    exception: Optional[Exception] = None
    traceback_text: str = ""


def read_header(file_path: str, encoding: str, delimiter: str) -> List[str]:
    with open(file_path, "r", newline="", encoding=encoding) as f:
        reader = csv.reader(f, delimiter=delimiter)
        return next(reader, [])


class WriterCache:
    def __init__(self, max_open: int, created_set: set):
        self.max_open = max_open
        self._order: List[str] = []
        self._writers: Dict[str, Tuple[object, csv.writer]] = {}
        self._created_set = created_set

    def get_writer(self, key: str, file_path: str, encoding: str, delimiter: str, header: Optional[List[str]], on_exists: str, created_files: List[str], skipped_files: List[str]) -> Optional[csv.writer]:
        if key in self._writers:
            if key in self._order:
                self._order.remove(key)
            self._order.append(key)
            return self._writers[key][1]

        file_exists = os.path.exists(file_path)
        if file_exists and file_path not in self._created_set and on_exists == "skip":
            skipped_files.append(file_path)
            return None

        ensure_dir(os.path.dirname(file_path))
        if file_exists and file_path in self._created_set:
            mode = "a"
            header_to_write = None
        else:
            mode = "w"
            header_to_write = header
            self._created_set.add(file_path)
            created_files.append(file_path)
        f = open(file_path, mode, newline="", encoding=encoding)
        writer = csv.writer(f, delimiter=delimiter)
        if header_to_write:
            writer.writerow(header_to_write)
        self._writers[key] = (f, writer)
        self._order.append(key)
        while len(self._order) > self.max_open:
            evict_key = self._order.pop(0)
            handle, _ = self._writers.pop(evict_key)
            try:
                handle.close()
            except Exception:
                pass
        return writer

    def close_all(self) -> None:
        for handle, _ in self._writers.values():
            try:
                handle.close()
            except Exception:
                pass
        self._writers.clear()
        self._order.clear()


def split_csv_file(
    file_path: str,
    settings: SplitSettings,
    log_cb: Callable[[str], None],
    progress_cb: Callable[[float], None],
    cancel_cb: Callable[[], bool],
) -> FileResult:
    result = FileResult(file_path=file_path, success=True)
    file_size = os.path.getsize(file_path) if os.path.exists(file_path) else 0
    file_size = max(1, file_size)

    def update_progress(file_obj) -> None:
        try:
            ratio = min(1.0, file_obj.tell() / file_size)
            progress_cb(ratio)
        except Exception:
            pass

    try:
        with open(file_path, "r", newline="", encoding=settings.encoding) as f:
            reader = csv.reader(f, delimiter=settings.delimiter)
            header = next(reader, None)
            if header is None:
                return result
            header_row = header if settings.keep_header else None

            created_set: set = set()
            if settings.mode == "row":
                if settings.row_mode == "fixed":
                    seq = 1
                    count = 0
                    row_index = 0
                    writer = None
                    handle = None
                    for row in reader:
                        if cancel_cb():
                            result.success = False
                            result.error_message = "用戶取消"
                            break
                        if count == 0:
                            seq_text = format_sequence(seq, settings.seq_format)
                            filename = f"{settings.prefix}{seq_text}{settings.suffix}.csv"
                            out_file = os.path.join(settings.output_dir, filename)
                            if os.path.exists(out_file) and settings.on_exists == "skip":
                                result.skipped_files.append(out_file)
                                writer = None
                                handle = None
                            else:
                                ensure_dir(settings.output_dir)
                                handle = open(out_file, "w", newline="", encoding=settings.encoding)
                                writer = csv.writer(handle, delimiter=settings.delimiter)
                                if header_row:
                                    writer.writerow(header_row)
                                result.created_files.append(out_file)
                            seq += 1
                        if writer:
                            writer.writerow(row)
                        count += 1
                        row_index += 1
                        if count >= settings.rows_per_file:
                            count = 0
                            if handle:
                                try:
                                    handle.close()
                                except Exception:
                                    pass
                            writer = None
                            handle = None
                        if row_index % 2000 == 0:
                            update_progress(f)
                    if handle:
                        try:
                            handle.close()
                        except Exception:
                            pass
                    update_progress(f)
                else:
                    if settings.unique_column not in header:
                        raise ValueError(f"列名不存在: {settings.unique_column}")
                    col_index = header.index(settings.unique_column)
                    cache = WriterCache(settings.max_open_files, created_set)
                    skipped_values: set = set()
                    for row in reader:
                        if cancel_cb():
                            result.success = False
                            result.error_message = "用戶取消"
                            break
                        key = sanitize_filename(row[col_index] if col_index < len(row) else "")
                        if key in skipped_values:
                            continue
                        filename = f"{settings.prefix}{key}{settings.suffix}.csv"
                        out_file = os.path.join(settings.output_dir, filename)
                        writer = cache.get_writer(key, out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                        if writer is None:
                            skipped_values.add(key)
                            continue
                        writer.writerow(row)
                        if len(result.created_files) % 100 == 0:
                            update_progress(f)
                    cache.close_all()
                    update_progress(f)
            else:
                if settings.col_column not in header:
                    raise ValueError(f"列名不存在: {settings.col_column}")
                col_index = header.index(settings.col_column)
                cache = WriterCache(settings.max_open_files, created_set)
                value_state: Dict[str, Dict[str, int]] = {}
                skipped_values: set = set()
                for row in reader:
                    if cancel_cb():
                        result.success = False
                        result.error_message = "用戶取消"
                        break
                    value = sanitize_filename(row[col_index] if col_index < len(row) else "")
                    if value in skipped_values:
                        continue
                    if settings.col_secondary_split:
                        state = value_state.setdefault(value, {"seq": 1, "count": 0})
                        if state["count"] == 0:
                            seq_text = format_sequence(state["seq"], settings.seq_format)
                            filename = f"{settings.prefix}{value}_{seq_text}{settings.suffix}.csv"
                            out_file = os.path.join(settings.output_dir, filename)
                            writer = cache.get_writer(f"{value}:{state['seq']}", out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                            if writer is None:
                                skipped_values.add(value)
                                continue
                            state["seq"] += 1
                        else:
                            writer = cache.get_writer(f"{value}:{state['seq']-1}", os.path.join(settings.output_dir, f"{settings.prefix}{value}_{format_sequence(state['seq']-1, settings.seq_format)}{settings.suffix}.csv"), settings.encoding, settings.delimiter, None, settings.on_exists, result.created_files, result.skipped_files)
                        if writer:
                            writer.writerow(row)
                        state["count"] += 1
                        if state["count"] >= settings.col_secondary_rows:
                            state["count"] = 0
                    else:
                        filename = f"{settings.prefix}{value}{settings.suffix}.csv"
                        out_file = os.path.join(settings.output_dir, filename)
                        writer = cache.get_writer(value, out_file, settings.encoding, settings.delimiter, header_row, settings.on_exists, result.created_files, result.skipped_files)
                        if writer is None:
                            skipped_values.add(value)
                            continue
                        writer.writerow(row)
                    if len(result.created_files) % 100 == 0:
                        update_progress(f)
                cache.close_all()
                update_progress(f)
    except UnicodeDecodeError as e:
        result.success = False
        result.error_message = f"編碼錯(cuò)誤: {e}"
        result.exception = e
        result.traceback_text = traceback.format_exc()
    except Exception as e:
        result.success = False
        result.error_message = str(e)
        result.exception = e
        result.traceback_text = traceback.format_exc()

    if result.exception and isinstance(result.exception, OSError) and getattr(result.exception, "errno", None) == errno.ENOSPC:
        log_cb(f"{timestamp()} 磁盤(pán)空間不足: {result.exception}")
    return result


class Worker(QObject):
    overall_progress = pyqtSignal(int)
    file_progress = pyqtSignal(int)
    log = pyqtSignal(str)
    finished = pyqtSignal(dict)
    show_error = pyqtSignal(str, str)

    def __init__(self, files: List[str], settings: SplitSettings):
        super().__init__()
        self.files = files
        self.settings = settings
        self._cancel = False
        self._created_files: List[str] = []

    def cancel(self) -> None:
        self._cancel = True

    def _is_cancelled(self) -> bool:
        return self._cancel

    def run(self) -> None:
        total_size = sum(max(1, os.path.getsize(p)) for p in self.files)
        processed_size = 0
        success_count = 0
        fail_count = 0

        for file_path in self.files:
            if self._cancel:
                break
            file_size = max(1, os.path.getsize(file_path))
            self.log.emit(f"{timestamp()} 開(kāi)始處理: {file_path}")

            def log_cb(msg: str) -> None:
                self.log.emit(msg)

            def progress_cb(ratio: float) -> None:
                self.file_progress.emit(int(ratio * 100))
                overall_ratio = min(1.0, (processed_size + ratio * file_size) / total_size)
                self.overall_progress.emit(int(overall_ratio * 100))

            result = split_csv_file(file_path, self.settings, log_cb, progress_cb, self._is_cancelled)
            self._created_files.extend(result.created_files)
            if result.success:
                success_count += 1
                self.log.emit(f"{timestamp()} 完成: {file_path} 輸出 {len(result.created_files)} 個(gè)文件")
            else:
                fail_count += 1
                detail = result.error_message
                if result.traceback_text:
                    detail = f"{result.error_message}\n{result.traceback_text}"
                self.log.emit(f"{timestamp()} 失敗: {file_path}\n{detail}")
                if isinstance(result.exception, OSError) and getattr(result.exception, "errno", None) == errno.ENOSPC:
                    self.show_error.emit("磁盤(pán)空間不足", detail)
            processed_size += file_size
            self.file_progress.emit(100)
            self.overall_progress.emit(int(min(1.0, processed_size / total_size) * 100))

        summary = {
            "success": success_count,
            "failed": fail_count,
            "cancelled": self._cancel,
        }
        if self._cancel:
            for path in self._created_files:
                try:
                    if os.path.exists(path):
                        os.remove(path)
                except Exception:
                    pass
        self.finished.emit(summary)


class FileListWidget(QListWidget):
    def __init__(self):
        super().__init__()
        self.setSelectionMode(QListWidget.ExtendedSelection)
        self.setAcceptDrops(True)

    def dragEnterEvent(self, event):
        if event.mimeData().hasUrls():
            event.acceptProposedAction()
        else:
            super().dragEnterEvent(event)

    def dropEvent(self, event):
        if event.mimeData().hasUrls():
            for url in event.mimeData().urls():
                path = url.toLocalFile()
                if path and path.lower().endswith(".csv"):
                    self.add_file(path)
            event.acceptProposedAction()
        else:
            super().dropEvent(event)

    def add_file(self, path: str) -> None:
        for i in range(self.count()):
            if self.item(i).text() == path:
                return
        self.addItem(QListWidgetItem(path))

    def files(self) -> List[str]:
        return [self.item(i).text() for i in range(self.count())]


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()
        self.setWindowTitle("CSV 拆分工具 作者:小莊-Python辦公")
        self.resize(1000, 720)

        self.file_list = FileListWidget()
        self.add_btn = QPushButton("添加文件")
        self.remove_btn = QPushButton("移除選中")
        self.clear_btn = QPushButton("清空列表")

        self.mode_row = QRadioButton("按行拆分")
        self.mode_col = QRadioButton("按列拆分")
        self.mode_row.setChecked(True)
        self.mode_group = QButtonGroup()
        self.mode_group.addButton(self.mode_row)
        self.mode_group.addButton(self.mode_col)

        self.stack = QStackedWidget()
        self.row_panel = self._build_row_panel()
        self.col_panel = self._build_col_panel()
        self.stack.addWidget(self.row_panel)
        self.stack.addWidget(self.col_panel)

        self.prefix_input = QLineEdit("part_")
        self.suffix_input = QLineEdit("")
        self.seq_input = QLineEdit("0001")
        self.keep_header = QCheckBox("保留表頭")
        self.keep_header.setChecked(True)

        self.output_dir_input = QLineEdit("")
        self.output_browse_btn = QPushButton("選擇輸出目錄")
        self.delimiter_combo = QComboBox()
        self.delimiter_combo.addItems([",", ";", "\\t", "|"])
        self.encoding_combo = QComboBox()
        self.encoding_combo.addItems(["UTF-8", "UTF-8-sig", "GBK"])
        self.exists_overwrite = QRadioButton("覆蓋")
        self.exists_skip = QRadioButton("跳過(guò)")
        self.exists_overwrite.setChecked(True)

        self.start_btn = QPushButton("開(kāi)始")
        self.cancel_btn = QPushButton("取消")
        self.cancel_btn.setEnabled(False)
        self.export_log_btn = QPushButton("導(dǎo)出日志")

        self.total_progress = QProgressBar()
        self.file_progress = QProgressBar()
        self.log_text = QTextEdit()
        self.log_text.setReadOnly(True)
        self.log_text.setFont(QFont("Consolas", 10))

        layout = QVBoxLayout()
        layout.addWidget(self._build_files_group())
        layout.addWidget(self._build_mode_group())
        layout.addWidget(self._build_naming_group())
        layout.addWidget(self._build_output_group())
        layout.addWidget(self._build_progress_group())
        layout.addWidget(self._build_log_group())

        container = QWidget()
        container.setLayout(layout)
        self.setCentralWidget(container)

        self.add_btn.clicked.connect(self._add_files)
        self.remove_btn.clicked.connect(self._remove_selected)
        self.clear_btn.clicked.connect(self.file_list.clear)
        self.mode_row.toggled.connect(self._switch_mode)
        self.mode_col.toggled.connect(self._switch_mode)
        self.output_browse_btn.clicked.connect(self._choose_output_dir)
        self.start_btn.clicked.connect(self._start)
        self.cancel_btn.clicked.connect(self._cancel)
        self.export_log_btn.clicked.connect(self._export_log)
        self.delimiter_combo.currentIndexChanged.connect(self._refresh_headers)
        self.encoding_combo.currentIndexChanged.connect(self._refresh_headers)
        self.file_list.itemSelectionChanged.connect(self._refresh_headers)

        self.worker_thread: Optional[QThread] = None
        self.worker: Optional[Worker] = None

    def _build_files_group(self) -> QGroupBox:
        group = QGroupBox("文件選擇")
        layout = QVBoxLayout()
        btn_layout = QHBoxLayout()
        btn_layout.addWidget(self.add_btn)
        btn_layout.addWidget(self.remove_btn)
        btn_layout.addWidget(self.clear_btn)
        layout.addLayout(btn_layout)
        layout.addWidget(self.file_list)
        group.setLayout(layout)
        return group

    def _build_mode_group(self) -> QGroupBox:
        group = QGroupBox("拆分模式")
        layout = QVBoxLayout()
        mode_layout = QHBoxLayout()
        mode_layout.addWidget(self.mode_row)
        mode_layout.addWidget(self.mode_col)
        layout.addLayout(mode_layout)
        layout.addWidget(self.stack)
        group.setLayout(layout)
        return group

    def _build_row_panel(self) -> QWidget:
        panel = QWidget()
        layout = QVBoxLayout()
        self.row_fixed = QRadioButton("按固定行數(shù)")
        self.row_unique = QRadioButton("按唯一列值")
        self.row_fixed.setChecked(True)
        self.row_group = QButtonGroup()
        self.row_group.addButton(self.row_fixed)
        self.row_group.addButton(self.row_unique)
        row_mode_layout = QHBoxLayout()
        row_mode_layout.addWidget(self.row_fixed)
        row_mode_layout.addWidget(self.row_unique)
        layout.addLayout(row_mode_layout)

        fixed_layout = QHBoxLayout()
        fixed_layout.addWidget(QLabel("每文件行數(shù)"))
        self.rows_per_file = QSpinBox()
        self.rows_per_file.setRange(1, 10_000_000)
        self.rows_per_file.setValue(100000)
        fixed_layout.addWidget(self.rows_per_file)
        layout.addLayout(fixed_layout)

        unique_layout = QHBoxLayout()
        unique_layout.addWidget(QLabel("列名"))
        self.row_unique_column = QComboBox()
        unique_layout.addWidget(self.row_unique_column)
        layout.addLayout(unique_layout)

        panel.setLayout(layout)
        return panel

    def _build_col_panel(self) -> QWidget:
        panel = QWidget()
        layout = QVBoxLayout()
        col_layout = QHBoxLayout()
        col_layout.addWidget(QLabel("列名"))
        self.col_column = QComboBox()
        col_layout.addWidget(self.col_column)
        layout.addLayout(col_layout)

        split_layout = QHBoxLayout()
        self.col_secondary = QCheckBox("每子文件再按行數(shù)拆分")
        split_layout.addWidget(self.col_secondary)
        split_layout.addWidget(QLabel("行數(shù)"))
        self.col_secondary_rows = QSpinBox()
        self.col_secondary_rows.setRange(1, 10_000_000)
        self.col_secondary_rows.setValue(100000)
        split_layout.addWidget(self.col_secondary_rows)
        layout.addLayout(split_layout)
        panel.setLayout(layout)
        return panel

    def _build_naming_group(self) -> QGroupBox:
        group = QGroupBox("命名規(guī)則")
        layout = QVBoxLayout()
        row1 = QHBoxLayout()
        row1.addWidget(QLabel("前綴"))
        row1.addWidget(self.prefix_input)
        row1.addWidget(QLabel("后綴"))
        row1.addWidget(self.suffix_input)
        layout.addLayout(row1)
        row2 = QHBoxLayout()
        row2.addWidget(QLabel("序號(hào)格式"))
        row2.addWidget(self.seq_input)
        row2.addWidget(self.keep_header)
        layout.addLayout(row2)
        group.setLayout(layout)
        return group

    def _build_output_group(self) -> QGroupBox:
        group = QGroupBox("輸出配置")
        layout = QVBoxLayout()
        row1 = QHBoxLayout()
        row1.addWidget(QLabel("輸出目錄"))
        row1.addWidget(self.output_dir_input)
        row1.addWidget(self.output_browse_btn)
        layout.addLayout(row1)

        row2 = QHBoxLayout()
        row2.addWidget(QLabel("分隔符"))
        row2.addWidget(self.delimiter_combo)
        row2.addWidget(QLabel("編碼"))
        row2.addWidget(self.encoding_combo)
        layout.addLayout(row2)

        row3 = QHBoxLayout()
        row3.addWidget(QLabel("已存在文件"))
        row3.addWidget(self.exists_overwrite)
        row3.addWidget(self.exists_skip)
        layout.addLayout(row3)
        group.setLayout(layout)
        return group

    def _build_progress_group(self) -> QGroupBox:
        group = QGroupBox("進(jìn)度")
        layout = QVBoxLayout()
        layout.addWidget(QLabel("總體進(jìn)度"))
        layout.addWidget(self.total_progress)
        layout.addWidget(QLabel("當(dāng)前文件進(jìn)度"))
        layout.addWidget(self.file_progress)
        row = QHBoxLayout()
        row.addWidget(self.start_btn)
        row.addWidget(self.cancel_btn)
        row.addWidget(self.export_log_btn)
        layout.addLayout(row)
        group.setLayout(layout)
        return group

    def _build_log_group(self) -> QGroupBox:
        group = QGroupBox("日志")
        layout = QVBoxLayout()
        layout.addWidget(self.log_text)
        group.setLayout(layout)
        return group

    def _switch_mode(self) -> None:
        self.stack.setCurrentIndex(0 if self.mode_row.isChecked() else 1)

    def _add_files(self) -> None:
        files, _ = QFileDialog.getOpenFileNames(self, "選擇 CSV 文件", "", "CSV Files (*.csv)")
        for path in files:
            self.file_list.add_file(path)
        self._refresh_headers()

    def _remove_selected(self) -> None:
        for item in self.file_list.selectedItems():
            row = self.file_list.row(item)
            self.file_list.takeItem(row)
        self._refresh_headers()

    def _choose_output_dir(self) -> None:
        path = QFileDialog.getExistingDirectory(self, "選擇輸出目錄")
        if path:
            self.output_dir_input.setText(path)

    def _delimiter_value(self) -> str:
        text = self.delimiter_combo.currentText()
        return "\t" if text == "\\t" else text

    def _refresh_headers(self) -> None:
        files = self.file_list.files()
        if not files:
            self.row_unique_column.clear()
            self.col_column.clear()
            return
        path = files[0]
        try:
            header = read_header(path, self.encoding_combo.currentText(), self._delimiter_value())
            self.row_unique_column.clear()
            self.col_column.clear()
            self.row_unique_column.addItems(header)
            self.col_column.addItems(header)
        except Exception as e:
            self.log_text.append(f"{timestamp()} 讀取表頭失敗: {e}")
            self.row_unique_column.clear()
            self.col_column.clear()

    def _build_settings(self) -> Optional[SplitSettings]:
        files = self.file_list.files()
        if not files:
            QMessageBox.warning(self, "提示", "請(qǐng)先選擇 CSV 文件")
            return None
        output_dir = self.output_dir_input.text().strip()
        if not output_dir:
            QMessageBox.warning(self, "提示", "請(qǐng)指定輸出目錄")
            return None
        delimiter = self._delimiter_value()
        encoding = self.encoding_combo.currentText()
        on_exists = "overwrite" if self.exists_overwrite.isChecked() else "skip"
        mode = "row" if self.mode_row.isChecked() else "col"
        row_mode = "fixed" if self.row_fixed.isChecked() else "unique"
        settings = SplitSettings(
            mode=mode,
            row_mode=row_mode,
            rows_per_file=self.rows_per_file.value(),
            unique_column=self.row_unique_column.currentText(),
            col_column=self.col_column.currentText(),
            col_secondary_split=self.col_secondary.isChecked(),
            col_secondary_rows=self.col_secondary_rows.value(),
            prefix=self.prefix_input.text(),
            suffix=self.suffix_input.text(),
            seq_format=self.seq_input.text() or "0001",
            keep_header=self.keep_header.isChecked(),
            delimiter=delimiter,
            encoding=encoding,
            output_dir=output_dir,
            on_exists=on_exists,
        )
        return settings

    def _start(self) -> None:
        settings = self._build_settings()
        if not settings:
            return
        files = self.file_list.files()
        ensure_dir(settings.output_dir)
        self.total_progress.setValue(0)
        self.file_progress.setValue(0)
        self.start_btn.setEnabled(False)
        self.cancel_btn.setEnabled(True)
        self.log_text.append(f"{timestamp()} 任務(wù)開(kāi)始,文件數(shù): {len(files)}")

        self.worker_thread = QThread()
        self.worker = Worker(files, settings)
        self.worker.moveToThread(self.worker_thread)
        self.worker_thread.started.connect(self.worker.run)
        self.worker.log.connect(self.log_text.append)
        self.worker.file_progress.connect(self.file_progress.setValue)
        self.worker.overall_progress.connect(self.total_progress.setValue)
        self.worker.finished.connect(self._finish)
        self.worker.show_error.connect(self._show_error)
        self.worker_thread.start()

    def _finish(self, summary: dict) -> None:
        if self.worker_thread:
            self.worker_thread.quit()
            self.worker_thread.wait()
        self.start_btn.setEnabled(True)
        self.cancel_btn.setEnabled(False)
        if summary.get("cancelled"):
            QMessageBox.information(self, "已取消", "任務(wù)已取消,臨時(shí)文件已清理")
        else:
            QMessageBox.information(
                self,
                "完成",
                f"成功: {summary.get('success', 0)},失敗: {summary.get('failed', 0)}",
            )

    def _cancel(self) -> None:
        if self.worker:
            self.worker.cancel()
            self.log_text.append(f"{timestamp()} 用戶取消任務(wù)")
            self.cancel_btn.setEnabled(False)

    def _show_error(self, title: str, detail: str) -> None:
        QMessageBox.critical(self, title, detail)

    def _export_log(self) -> None:
        path, _ = QFileDialog.getSaveFileName(self, "導(dǎo)出日志", "log.txt", "Text Files (*.txt)")
        if not path:
            return
        try:
            with open(path, "w", encoding="utf-8") as f:
                f.write(self.log_text.toPlainText())
            QMessageBox.information(self, "成功", "日志已導(dǎo)出")
        except Exception as e:
            QMessageBox.critical(self, "失敗", str(e))


def main() -> None:
    app = QApplication(sys.argv)
    window = MainWindow()
    window.show()
    sys.exit(app.exec_())


if __name__ == "__main__":
    main()

可擴(kuò)展方向

如果繼續(xù)增強(qiáng),可考慮:

  • 增加“預(yù)覽前 N 行”功能
  • 支持保存與加載配置
  • 增加“輸出文件壓縮”為 zip
  • 提供“拆分后統(tǒng)計(jì)報(bào)表”

結(jié)語(yǔ)

這個(gè)工具的核心理念是“穩(wěn)定、低內(nèi)存、高可用”。通過(guò)標(biāo)準(zhǔn)庫(kù) csv 與 PyQt5 的組合,達(dá)成了性能與交互的平衡,并保持項(xiàng)目結(jié)構(gòu)簡(jiǎn)單、易維護(hù)。對(duì)于日常辦公與大規(guī)模數(shù)據(jù)拆分,已具備可直接使用的工程級(jí)質(zhì)量。

以上就是Python實(shí)現(xiàn)批量將CSV按行或按列拆分的詳細(xì)內(nèi)容,更多關(guān)于Python拆分CSV的資料請(qǐng)關(guān)注腳本之家其它相關(guān)文章!

相關(guān)文章

  • Pytorch+PyG實(shí)現(xiàn)EdgeCNN過(guò)程示例詳解

    Pytorch+PyG實(shí)現(xiàn)EdgeCNN過(guò)程示例詳解

    這篇文章主要為大家介紹了Pytorch+PyG實(shí)現(xiàn)EdgeCNN過(guò)程示例詳解,有需要的朋友可以借鑒參考下,希望能夠有所幫助,祝大家多多進(jìn)步,早日升職加薪
    2023-04-04
  • 一文學(xué)會(huì)如何將Python打包后的exe還原成.py

    一文學(xué)會(huì)如何將Python打包后的exe還原成.py

    反編譯的第一步就是要將exe文件轉(zhuǎn)換成py文件,下面這篇文章主要給大家介紹了如何通過(guò)一文學(xué)會(huì)將Python打包后的exe還原成.py的相關(guān)資料,文中通過(guò)實(shí)例代碼介紹的非常詳細(xì),需要的朋友可以參考下
    2022-11-11
  • Python中Dict兩種實(shí)現(xiàn)的原理詳解

    Python中Dict兩種實(shí)現(xiàn)的原理詳解

    在Python中,?Dict是一系列由鍵和值配對(duì)組成的元素的集合,?它是一個(gè)可變?nèi)萜髂P停梢源鎯?chǔ)任意類型對(duì)象。本文主要介紹了Dict兩種實(shí)現(xiàn)的原理,感興趣的可以了解一下
    2023-03-03
  • python列表刪除元素的三種實(shí)現(xiàn)方法

    python列表刪除元素的三種實(shí)現(xiàn)方法

    本文主要介紹了python列表刪除元素的三種實(shí)現(xiàn)方法,主要包括pop方法,remove方法,del方法這三種,具有一定的參考價(jià)值,感興趣的可以了解一下
    2024-01-01
  • Python自動(dòng)化辦公之Word文檔的創(chuàng)建與生成

    Python自動(dòng)化辦公之Word文檔的創(chuàng)建與生成

    這篇文章主要為大家詳細(xì)介紹了如何通過(guò)python腳本來(lái)自動(dòng)生成一個(gè)?word文檔,文中的示例代碼講解詳細(xì),感興趣的小伙伴可以了解一下
    2022-05-05
  • python實(shí)現(xiàn)上傳下載文件功能

    python實(shí)現(xiàn)上傳下載文件功能

    這篇文章主要為大家詳細(xì)介紹了python實(shí)現(xiàn)上傳下載文件功能,文中示例代碼介紹的非常詳細(xì),具有一定的參考價(jià)值,感興趣的小伙伴們可以參考一下
    2017-07-07
  • Python實(shí)現(xiàn)在Word中創(chuàng)建,讀取和刪除列表詳解

    Python實(shí)現(xiàn)在Word中創(chuàng)建,讀取和刪除列表詳解

    在Word中,列表是一種用于組織和呈現(xiàn)信息的有效工具,這篇文章將探討一下如何使用Python在Word文檔中創(chuàng)建,讀取和刪除列表,需要的可以了解下
    2025-03-03
  • Python基礎(chǔ)知識(shí)點(diǎn) 初識(shí)Python.md

    Python基礎(chǔ)知識(shí)點(diǎn) 初識(shí)Python.md

    在本篇文章中我們給大家總結(jié)了關(guān)于Python基礎(chǔ)知識(shí)點(diǎn),通過(guò)初識(shí)Python.md的相關(guān)內(nèi)容分享給Python初學(xué)者,一起來(lái)看下吧。
    2019-05-05
  • Python文件的寫(xiě)入操作write與writelines方法詳解

    Python文件的寫(xiě)入操作write與writelines方法詳解

    在Python編程中,文件操作是一個(gè)非常重要的基礎(chǔ)知識(shí),無(wú)論是處理數(shù)據(jù)、保存配置還是記錄日志,我們都需要頻繁地進(jìn)行文件讀寫(xiě)操作,今天我們就來(lái)深入探討Python中的文件寫(xiě)入操作,特別是write()和writelines()這兩個(gè)核心方法,需要的朋友可以參考下
    2026-05-05
  • python裝飾器深入學(xué)習(xí)

    python裝飾器深入學(xué)習(xí)

    這篇文章主要深入學(xué)習(xí)了python裝飾器的相關(guān)資料,什么是裝飾器?裝飾器遵循的原則等,具有一定的參考價(jià)值,感興趣的小伙伴們可以參考一下
    2018-04-04

最新評(píng)論

揭阳市| 哈尔滨市| 武夷山市| 通渭县| 广灵县| 佛教| 榕江县| 绩溪县| 曲靖市| 青田县| 尼玛县| 高密市| 浦江县| 新蔡县| 仪陇县| 宜宾市| 巴东县| 昌图县| 沧州市| 安陆市| 格尔木市| 临猗县| 乌拉特中旗| 琼中| 应城市| 文安县| 新竹市| 梅河口市| 阿拉善左旗| 嘉善县| 崇文区| 古丈县| 保康县| 横峰县| 屏东市| 天等县| 永安市| 中阳县| 桐庐县| 台中市| 翁牛特旗|