Steampipe 2.4.9 配置参数详解
Steampipe v2.4.9 配置体系实测详解:~/.steampipe/config 下 default.spc 的 database/general/plugin_manager 参数、v2 新版 connection 单 label 语法与 aggregator 聚合、workspaces 档案、配置优先级与生效方式,附 steampipe.rc 失效等排障实录。
Steampipe 2.4.9 配置参数详解
目标:把 Steampipe v2.4.9 的配置文件体系讲透——哪些文件放哪、每个参数管什么、改完怎么生效,以及 v2 相对旧版语法的两处硬变更(connection 单 label、steampipe.rc 失效)。安装与首个服务见 《Ubuntu 22.04 安装 Steampipe 2.4.9 指南》。
一、环境信息
| 项目 | 值 |
|---|---|
| 操作系统 | Ubuntu 22.04.5 LTS (Jammy Jellyfish) |
| 架构 | x86_64 |
| 软件版本 | Steampipe v2.4.9(内嵌 PostgreSQL 14.19) |
| 配置日期 | 2026-11-03 |
| 运行用户 | 普通用户 admin(root 被拒绝运行) |
| workspace | ~/.steampipe(默认) |
二、配置体系总览
Steampipe v2 的配置全部以 HCL 语法写入 ~/.steampipe/config/ 下的 *.spc 文件,启动时加载该目录全部 .spc 文件(官方文档 Configuration Files,2026-11-03 实查)。实测目录布局:
~/.steampipe/config/
├── default.spc # 全局 options(database/general/plugin_manager)
├── hackernews.spc # 插件安装时自动生成的默认连接文件
├── steampipe.spc # 同上(steampipe 插件)
├── hn.spc # 自建的命名连接 + 聚合连接
└── *.spc.sample # 初始化生成的样例(default.spc.sample 等)
三条实测关键规则:
- 优先级:CLI 参数 > .spc 配置。
steampipe service start --database-listen的默认值network会覆盖default.spc中的listen = "local"(实测:systemd 单元未带该 flag 时 9193 监听所有网卡;显式传--database-listen local后收紧为仅回环)。 - 改配置必须重启服务:
steampipe service restart。实测新建命名连接hn后不重启,plugin list与查询均不可见;重启后立即可用。 steampipe.rc在 v2.4.9 不生效:旧版文档的steampipe.rc(如database_port = 9194)实测被完全忽略——移除 default.spc 后服务仍按内置默认 9193 启动(排障实录见第十节)。官方文档站对应页面已 404,v2 配置已全面收敛到.spc体系。
三、options "database"(内嵌 PostgreSQL 行为)
初始化生成的 default.spc.sample 原文(节选,注释即官方文档):
# Steampipe specific configuration
options "database" {
# Sets the port that the database service listens on. Default 9193
# port = 9193
# Listen on localhost (local), tailnet (tailnet), any (network)
# Default: network - use "local" to accept connections only from localhost
# listen = "network"
# Set to true to enable dashboard service
# dashboard = false
# Create a steampipe foreign table for each connection
# Set to false to disable creation of the steampipe foreign table
# foreign_tables_enabled = true
# Enable TLS encryption
# ssl = false
# Set the database password - this must be a strong password
# if not set, a random password will be generated
# password = "mySuperSecretPassword"
# Set the search path for the database - this is a comma separated list of
# schemas to search for unqualified table names
# search_path = ""
# Enable query result caching
# cache = true
# Set the cache TTL for query results - defaults to 300 seconds
# cache_max_ttl = 300
# Set the shared_buffers for the database
# shared_buffers = "64MB"
# Set the effective_cache_size for the database
# effective_cache_size = "512MB"
# Set the work_mem for the database
# work_mem = "4MB"
# Set the minimum duration in milliseconds before a query is logged
# log_min_duration = 1000
# Enable logging of query stats
# log_query_stats = true
# Enable logging of database connections
# log_connections = false
# Enable logging of database disconnections
# log_disconnections = false
}
实测生效要点:
| 参数 | 默认 | 实测表现 |
|---|---|---|
| port | 9193 | service status 回显 Port: 9193,psql 可直连 |
| listen | network | 设 local 后 Host(s) 仅剩 localhost/127.0.0.1/::1,从容器网段 IP 连接被 Connection refused |
| ssl | false(但初始化后实测为 true) | service status 显示 SSL Enabled: true |
| password | 随机(存 internal/.passwd) | 演示环境统一 q1w2e3r4;也可 service start --database-password |
| cache / cache_max_ttl | true / 300s | 写入 default.spc 后服务正常重启生效 |
| shared_buffers 等 PG 内存参数 | 64MB/512MB/4MB | 同 PG 语义,控制内嵌实例资源 |
四、options "general" 与 options "plugin_manager"
options "general" {
# log level - error, warn, info, debug, trace
log_level = "info"
# number of log files to keep
log_max_backups = 30
# maximum size of a log file in MB
log_file_max_size_mb = 100
# whether to log plugin stderr output
log_plugin_stderr = false
# timeout in seconds for plugin startup or queries (set to 0 to disable)
plugin_timeout = 300
query_timeout = 300
# enable update check (stable stream only)
update_check = true
# telemetry level - none, errors, standard, all
telemetry = "standard"
# index size in MB
index_size = 128
}
options "plugin_manager" {
# maximum number of concurrent plugin connections
max_concurrency = 12
}
plugin_manager.max_concurrency 控制插件并发上限,是跨源大查询的第一调优开关;query_timeout/plugin_timeout 与 steampipe query --query-timeout 参数同源(CLI 参数优先)。
五、connection 块(数据源连接)
5.1 v2 语法:单 label
v2.4.9 的 connection 块只接受 1 个 label,插件由块内 plugin 参数指定。沿用 v0.x 三 label 写法会被直接拒绝(实测回显):
# v0.x 旧语法(已失效)
connection "hackernews" "hackernews" "hn" {
}
Error: Internal Error: Failed to load config: Extraneous label for connection:
Only 1 labels (name) are expected for connection blocks.
(/home/admin/.steampipe/config/hn.spc:1,1-30)
v2 正确写法与实测:
# ~/.steampipe/config/hn.spc
connection "hn" {
plugin = "hackernews"
}
$ steampipe service restart
$ steampipe query "select count(*) as hn_top_stories from hn.hackernews_top;"
+----------------+
| hn_top_stories |
+----------------+
| 500 |
+----------------+
每个连接对应一个 Postgres schema(hn),连接名即 schema 前缀。
5.2 通用参数(官方 connection 文档,2026-11-03 实查)
| 参数 | 默认 | 说明 |
|---|---|---|
| plugin | 无 | 插件版本串(如 hackernews、hackernews@1.2.0、acme/plugin@stream)或 plugin 块引用 |
| type | plugin | plugin 或 aggregator |
| import_schema | enabled | 设为 disabled 不建 schema(仍可被聚合器查询),连接数多时可加速启动 |
| {插件自有参数} | 各异 | 如 hackernews 的 max_items;凭证类参数见各插件 Hub 页 |
插件安装时自动生成的默认连接文件(实测 hackernews.spc 原文):
connection "hackernews" {
plugin = "hackernews"
# The maximum number of items to return for list and search queries.
# max_items = 100
}
5.3 aggregator 聚合连接(实测)
多连接聚合成单一查询视图(典型用途:多账号/多区域)。v2 语法必须带 plugin,成员用 connections 列表:
connection "hn_all" {
plugin = "hackernews"
type = "aggregator"
connections = ["hn", "hackernews"]
}
实测:重启后 plugin list 显示 Connections: hn,hn_all,hackernews;select count(*) from hn_all.hackernews_top; 返回 1,000(两连接各 500 行合并):
+----------+
| agg_rows |
+----------+
| 1,000 |
+----------+
💡 技术拓展:聚合器与 UNION ALL 视图、查询联邦引擎(Trino)解决同一问题「多源同构数据并查」——聚合器是插件内轻量方案(查询时展开到各连接,受 API 限速放大影响,官方文档明确提醒
select *会成倍放大 API 调用);Trino 面向湖仓大扫描;UNION 视图需手工维护。小规模多账号审计用 aggregator 最省事。
5.4 非限定表名与 search_path
未加前缀的表名按 Postgres search_path 解析:默认 public 在前,其余连接 schema 按字母序(官方 Managing Connections 文档,2026-11-03)。实测 hn 与默认 hackernews 并存时,select ... from hackernews_top(不带前缀)命中字母序靠前者。可用 options "database" search_path 或 --search-path 调整。
六、workspaces 档案(workspaces.spc)
初始化样例原文(workspaces.spc.sample):
# workspace "profile_name" {
# # database
# database_listen = "network"
# database_port = 9193
# database_password = "mySuperSecretPassword"
# database_search_path = ""
# database_ssl = false
# foreign_tables_enabled = true
# dashboard = false
# dashboard_database = 9193
# dashboard_listen = "network"
# dashboard_port = 9193
# dashboard_watch = false
#
# # general
# index_size = 128
# log_level = "info"
# plugin_timeout = 300
# query_timeout = 300
# update_check = true
# telemetry = "standard"
#
# # plugin manager
# plugin_manager_max_concurrency = 12
# }
启用方式:去掉注释中的 profile_name 改名(如 prod),用 steampipe --workspace prod <命令> 或环境变量 STEAMPIPE_WORKSPACE=prod 选择档案;--install-dir 可整体迁移 workspace 根目录。适合「同一台机器多套端口/凭证组合」隔离。
七、插件源与镜像(实测)
- 插件安装源为 Hub:
plugin list显示完整引用hub.steampipe.io/plugins/turbot/hackernews@latest,即「registry/组织/插件@流」结构; - 支持本地文件安装:
steampipe plugin install /path/steampipe-plugin-foo.tar.gz(plugin install --help原文),离线环境可先在有网机器拉包再分发; - 版本锁定:安装时
plugin@x.y.z或plugin@stream; - 实测风险信号:steampipe 插件查 registry 元数据走
artifactregistry.googleapis.com,2026-11-03 当日直连 TLS 握手超时、经代理 403,steampipe_registry_plugin表查询不稳定;插件安装本身走 Hub CDN,不受影响(hackernews/steampipe 均正常安装)。
八、运行时内部文件
~/.steampipe/internal/
├── connection.json # 服务运行时的连接串(服务停止时为 {})
├── .passwd # 内嵌库随机口令(未显式设 password 时)
├── update_check.json # 版本检查缓存(实测含 current_version/latest_version)
└── versions.json # 组件版本清单
~/.steampipe/logs/ # CLI/服务/插件日志(log_level=info 实测产出)
九、典型场景配置示例
① 单机安全收紧(生产推荐)——仅本机可连 + 固定口令 + 关遥测与更新检查:
# default.spc
options "database" {
listen = "local"
password = "q1w2e3r4"
}
options "general" {
telemetry = "none"
update_check = false
}
systemd 单元里同步显式传参(CLI flag 默认值会覆盖配置,见第二节):
ExecStart=/usr/local/bin/steampipe service start --foreground --database-listen local --database-password q1w2e3r4
② 多账号聚合审计(aws 类插件的通用形态,hackernews 双连接已实测等价验证):
connection "acct_prod" { plugin = "aws", profile = "prod" }
connection "acct_test" { plugin = "aws", profile = "test" }
connection "aws_all" {
plugin = "aws"
type = "aggregator"
connections = ["acct_prod", "acct_test"]
}
# select * from aws_all.aws_s3_bucket
③ 大查询性能:cache_max_ttl 上调 + plugin_manager.max_concurrency 上调 + 不需要的连接 import_schema = "disabled"。
十、常见问题
steampipe.rc写了不生效:v2.4.9 实测忽略该文件(database_port = 9194无效,服务仍按 default.spc/内置默认 9193)。所有配置走.spc。default.spc的listen="local"没生效:被service start的--database-listen默认值network覆盖;显式传 flag 或不带该 flag 由配置决定(实测两种行为均复现)。- 改 .spc 后查询报
relation ... does not exist:未重启服务。steampipe service restart后生效。 - connection 三 label 写法报
Extraneous label for connection:v2 只收 1 个 label,插件写进块内plugin = "..."。 - aggregator 报
The argument "plugin" is required:v2 聚合块必须显式带plugin,成员列表键为connections(非旧版members)。 - registry 表查询超时:
artifactregistry.googleapis.com不稳定(实测 DeadlineExceeded/403),不影响 Hub 插件安装;统计类需求以 Hub 网页为准。
十一、参考链接
- 官方 Configuration Files:https://steampipe.io/docs/reference/config-files
- 官方 Connection:https://steampipe.io/docs/reference/config-files/connection
- 官方 Managing Connections:https://steampipe.io/docs/managing/connections
- Steampipe Hub:https://hub.steampipe.io/plugins
- 站内:《Ubuntu 22.04 安装 Steampipe 2.4.9 指南》、《Steampipe 介绍与选型指南》、《CloudQuery 6.50 配置参数详解》