基於 mysql-connector-python 驅動,示範三種結果集獲取方法的適用場景與記憶體效能權衡。
pip install mysql-connector-python
# 配置檔 python_mysql_dbconfig.py
def read_db_config():
return {
'host': 'localhost',
'database': 'testdb',
'user': 'root',
'password': 'password'
}
from mysql.connector import MySQLConnection, Error
from python_mysql_dbconfig import read_db_config
def query_with_fetchone():
try:
dbconfig = read_db_config()
conn = MySQLConnection(**dbconfig)
cursor = conn.cursor()
cursor.execute("SELECT * FROM books")
row = cursor.fetchone()
while row is not None:
print(row)
row = cursor.fetchone()
except Error as e:
print(e)
finally:
cursor.close()
conn.close()
if __name__ == '__main__':
query_with_fetchone()
適用: 超大結果集、串流處理、記憶體受限環境
def query_with_fetchall():
try:
dbconfig = read_db_config()
conn = MySQLConnection(**dbconfig)
cursor = conn.cursor()
cursor.execute("SELECT * FROM books")
rows = cursor.fetchall()
print('Total Row(s):', cursor.rowcount)
for row in rows:
print(row)
except Error as e:
print(e)
finally:
cursor.close()
conn.close()
適用: 結果集較小 (如 < 10 萬行)、需隨機存取、後續多次遍歷
def iter_row(cursor, size=10):
"""生成器:分批 yield 行資料"""
while True:
rows = cursor.fetchmany(size)
if not rows:
break
for row in rows:
yield row
def query_with_fetchmany():
try:
dbconfig = read_db_config()
conn = MySQLConnection(**dbconfig)
cursor = conn.cursor()
cursor.execute("SELECT * FROM books")
for row in iter_row(cursor, 10): # 每次取 10 行
print(row)
except Error as e:
print(e)
finally:
cursor.close()
conn.close()
適用: 中大型結果集、避免一次性載入 OOM、需控制批次大小
| 方法 | 記憶體 | 速度 | 適用場景 |
|---|---|---|---|
| fetchone | 極低 | 較慢 (多次往返) | 超大表、串流、ETL |
| fetchmany | 可控 (batch size) | 平衡 | 大多數生產查詢 |
| fetchall | 高 (全量) | 最快 (單次往返) | 小表、查詢快取 |
cursor = conn.cursor(buffered=True) 可客戶端緩衝結果集,支援 cursor.rowcount 即時可用cursor.executemany() 批次執行mysql.connector.pooling.MySQLConnectionPoolaiomysql / asyncmy (Python 3.7+)