postgresql insert into select無法使用并行查詢的解決
更新時間：2021年01月08日 10:40:23 作者：瀚高PG實驗室
這篇文章主要介紹了postgresql insert into select無法使用并行查詢的解決，文中通過示例代碼介紹的非常詳細，對大家的學習或者工作具有一定的參考學習價值，需要的朋友們下面隨著小編來一起學習學習吧
本文信息基于PG13.1。
從PG9.6開始支持并行查詢。PG11開始支持CREATE TABLE … AS、SELECT INTO以及CREATE MATERIALIZED VIEW的并行查詢。
先說結(jié)論：
換用create table as 或者select into或者導入導出。
首先跟蹤如下查詢語句的執(zhí)行計劃：
select count(*) from test t1,test1 t2 where t1.id = t2.id ;
postgres=# explain analyze select count(*) from test t1,test1 t2 where t1.id = t2.id ;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------
Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=683.246..715.324 rows=1 loops=1)
-> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=681.474..715.311 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=674.689..675.285 rows=1 loops=3)
-> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=447.799..645.689 rows=333333 loops=3)
Hash Cond: (t1.id = t2.id)
-> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.025..74.010 rows=333333 loops=3)
-> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=260.052..260.053 rows=333333 loops=3)
Buckets: 131072 Batches: 16 Memory Usage: 3520kB
-> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.032..104.804 rows=333333 loops=3)
Planning Time: 0.420 ms
Execution Time: 715.447 ms
(13 rows)
可以看到走了兩個Workers。
下邊看一下insert into select：
postgres=# explain analyze insert into va select count(*) from test t1,test1 t2 where t1.id = t2.id ;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------
Insert on va (cost=73228.00..73228.02 rows=1 width=4) (actual time=3744.179..3744.187 rows=0 loops=1)
-> Subquery Scan on "*SELECT*" (cost=73228.00..73228.02 rows=1 width=4) (actual time=3743.343..3743.352 rows=1 loops=1)
-> Aggregate (cost=73228.00..73228.01 rows=1 width=8) (actual time=3743.247..3743.254 rows=1 loops=1)
-> Hash Join (cost=30832.00..70728.00 rows=1000000 width=0) (actual time=1092.295..3511.301 rows=1000000 loops=1)
Hash Cond: (t1.id = t2.id)
-> Seq Scan on test t1 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.030..421.537 rows=1000000 loops=1)
-> Hash (cost=14425.00..14425.00 rows=1000000 width=4) (actual time=1090.078..1090.081 rows=1000000 loops=1)
Buckets: 131072 Batches: 16 Memory Usage: 3227kB
-> Seq Scan on test1 t2 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.021..422.768 rows=1000000 loops=1)
Planning Time: 0.511 ms
Execution Time: 3745.633 ms
(11 rows)
可以看到并沒有Workers的指示，沒有啟用并行查詢。
即使開啟強制并行，也無法走并行查詢。
postgres=# set force_parallel_mode =on;
SET
postgres=# explain analyze insert into va select count(*) from test t1,test1 t2 where t1.id = t2.id ;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------
Insert on va (cost=73228.00..73228.02 rows=1 width=4) (actual time=3825.042..3825.049 rows=0 loops=1)
-> Subquery Scan on "*SELECT*" (cost=73228.00..73228.02 rows=1 width=4) (actual time=3824.976..3824.984 rows=1 loops=1)
-> Aggregate (cost=73228.00..73228.01 rows=1 width=8) (actual time=3824.972..3824.978 rows=1 loops=1)
-> Hash Join (cost=30832.00..70728.00 rows=1000000 width=0) (actual time=1073.587..3599.402 rows=1000000 loops=1)
Hash Cond: (t1.id = t2.id)
-> Seq Scan on test t1 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.034..414.965 rows=1000000 loops=1)
-> Hash (cost=14425.00..14425.00 rows=1000000 width=4) (actual time=1072.441..1072.443 rows=1000000 loops=1)
Buckets: 131072 Batches: 16 Memory Usage: 3227kB
-> Seq Scan on test1 t2 (cost=0.00..14425.00 rows=1000000 width=4) (actual time=0.022..400.624 rows=1000000 loops=1)
Planning Time: 0.577 ms
Execution Time: 3825.923 ms
(11 rows)
原因在官方文檔有寫：
The query writes any data or locks any database rows. If a query contains a data-modifying operation either at the top level or within a CTE, no parallel plans for that query will be generated. As an exception, the commands CREATE TABLE … AS, SELECT INTO, and CREATE MATERIALIZED VIEW which create a new table and populate it can use a parallel plan.
解決方案有如下三種：
1.select into
postgres=# explain analyze select count(*) into vaa from test t1,test1 t2 where t1.id = t2.id ;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------
Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=742.736..774.923 rows=1 loops=1)
-> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=740.223..774.907 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=731.408..731.413 rows=1 loops=3)
-> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=489.880..700.830 rows=333333 loops=3)
Hash Cond: (t1.id = t2.id)
-> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.033..87.479 rows=333333 loops=3)
-> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=266.839..266.840 rows=333333 loops=3)
Buckets: 131072 Batches: 16 Memory Usage: 3520kB
-> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.058..106.874 rows=333333 loops=3)
Planning Time: 0.319 ms
Execution Time: 783.300 ms
(13 rows)
2.create table as
postgres=# explain analyze create table vb as select count(*) from test t1,test1 t2 where t1.id = t2.id ;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------
Finalize Aggregate (cost=34244.16..34244.17 rows=1 width=8) (actual time=540.120..563.733 rows=1 loops=1)
-> Gather (cost=34243.95..34244.16 rows=2 width=8) (actual time=537.982..563.720 rows=3 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Partial Aggregate (cost=33243.95..33243.96 rows=1 width=8) (actual time=526.602..527.136 rows=1 loops=3)
-> Parallel Hash Join (cost=15428.00..32202.28 rows=416667 width=0) (actual time=334.532..502.793 rows=333333 loops=3)
Hash Cond: (t1.id = t2.id)
-> Parallel Seq Scan on test t1 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.018..57.819 rows=333333 loops=3)
-> Parallel Hash (cost=8591.67..8591.67 rows=416667 width=4) (actual time=189.502..189.503 rows=333333 loops=3)
Buckets: 131072 Batches: 16 Memory Usage: 3520kB
-> Parallel Seq Scan on test1 t2 (cost=0.00..8591.67 rows=416667 width=4) (actual time=0.023..77.786 rows=333333 loops=3)
Planning Time: 0.189 ms
Execution Time: 565.448 ms
(13 rows)
3.或者通過導入導出的方式，例如：
psql -h localhost -d postgres -U postgres -c "select count(*) from test t1,test1 t2 where t1.id = t2.id " -o result.csv -A -t -F "," psql -h localhost -d postgres -U postgres -c "COPY va FROM 'result.csv' WITH (FORMAT CSV, DELIMITER ',', HEADER FALSE, ENCODING 'windows-1252')"
一些場景下也會比非并行快。
到此這篇關于postgresql insert into select無法使用并行查詢的解決的文章就介紹到這了,更多相關postgresql insert into select并行查詢內(nèi)容請搜索腳本之家以前的文章或繼續(xù)瀏覽下面的相關文章希望大家以后多多支持腳本之家！
您可能感興趣的文章:
- mssql insert into 和insert into select性能比較
- 解析MySQL中INSERT INTO SELECT的使用
- SELECT INTO 和 INSERT INTO SELECT 兩種表復制語句詳解(SQL數(shù)據(jù)庫和Oracle數(shù)據(jù)庫的區(qū)別)
- postgresql 13.1 insert into select并行查詢的實現(xiàn)
- SQL Server之SELECT INTO 和 INSERT INTO SELECT案例詳解
- 關于MySQL中“Insert into select“ 的死鎖情況分析
- mysql使用insert into select插入查出的數(shù)據(jù)
- mysql中insert?into...select語句優(yōu)化方式
- MySQL insert into select 主鍵沖突解決方案
- sql中select into和insert select的用法小結(jié)
相關文章
PostgreSQL中ON?CONFLICT的使用及一些擴展用法
Postgres?ON?CONFLICT是PostgreSQL數(shù)據(jù)庫中的一個功能,用于處理插入或更新數(shù)據(jù)時的沖突情況,下面這篇文章主要給大家介紹了關于PostgreSQL中ON?CONFLICT的使用及一些擴展用法的相關資料,需要的朋友可以參考下2024-06-06
PostgreSQL數(shù)據(jù)庫備份與恢復的四種辦法
在數(shù)據(jù)為王的時代,數(shù)據(jù)庫中存儲的信息堪稱企業(yè)的生命線,而PostgreSQL作為一款廣泛應用的開源數(shù)據(jù)庫,學會如何妥善進行備份與恢復操作,是每個開發(fā)者與運維人員必備的技能,今天,咱們就深入探究一下PostgreSQL相關的備份恢復策略,并附上豐富的代碼示例2025-01-01
postgresql 刪除重復數(shù)據(jù)案例詳解
這篇文章主要介紹了postgresql 刪除重復數(shù)據(jù)案例詳解,本篇文章通過簡要的案例,講解了該項技術的了解與使用,以下就是詳細內(nèi)容,需要的朋友可以參考下2021-08-08
查看postgresql數(shù)據(jù)庫用戶系統(tǒng)權(quán)限、對象權(quán)限的方法
這篇文章主要介紹了查看postgresql數(shù)據(jù)庫用戶系統(tǒng)權(quán)限、對象權(quán)限的方法,本文給大家介紹的非常詳細，對大家的學習或工作具有一定的參考借鑒價值，需要的朋友可以參考下2020-12-12
PGSQL查詢最近N天的數(shù)據(jù)及SQL語句實現(xiàn)替換字段內(nèi)容
PostgreSQL提供了WITH語句,允許你構(gòu)造用于查詢的輔助語句,下面這篇文章主要給大家介紹了關于PGSQL查詢最近N天的數(shù)據(jù)及SQL語句實現(xiàn)替換字段內(nèi)容的相關資料,文中通過實例代碼介紹的非常詳細,需要的朋友可以參考下2023-03-03
最近有這么一個工程,需要使用postgresql數(shù)據(jù)庫,在數(shù)據(jù)庫中的好幾個表中都需要時間戳這個字段,這篇文章主要給大家介紹了關于PostgreSQL自動更新時間戳的相關資料,需要的朋友可以參考下2021-11-11
PostgreSQL 提供了多種不同的復制表的方法，它們的差異在于是否需要復制表結(jié)構(gòu)或者數(shù)據(jù)，這篇文章主要介紹了PostgreSQL 復制表的 5 種方式,需要的朋友可以參考下2023-01-01
這篇文章主要介紹了PostgreSQL JSONB的匹配和交集問題，本文給大家介紹的非常詳細，對大家的學習或工作具有一定的參考借鑒價值，需要的朋友可以參考下2021-09-09
Postgresql中json和jsonb類型區(qū)別解析
在我們的業(yè)務開發(fā)中,可能會因為特殊【歷史,偷懶,防止表連接】經(jīng)常會有JSON或者JSONArray類的數(shù)據(jù)存儲到某列中,這個時候再PG數(shù)據(jù)庫中有兩種數(shù)據(jù)格式可以直接一對多或者一對一的映射對象,接下來通過本文介紹Postgresql中json和jsonb類型區(qū)別,需要的朋友可以參考下2024-06-06
詳解如何優(yōu)化在PostgreSQL中對于日期范圍的查詢
在 PostgreSQL 中,處理日期范圍的查詢是常見的操作,然而,如果不進行適當?shù)膬?yōu)化,這些查詢可能會導致性能問題,特別是在處理大型數(shù)據(jù)集時,本文章將詳細討論如何優(yōu)化在 PostgreSQL 中對于日期范圍的查詢,需要的朋友可以參考下2024-07-07