Apache ManifoldCF 2.31 常用 CLI 命令与工具详解
详解 Apache ManifoldCF 2.31 常用 CLI 与工具:combined 与多进程启停脚本、curl REST API 作业与连接管理、脚本引擎 run-script.sh、obfuscate.sh 密码混淆、jps/netstat 核对与 systemd 托管,命令与回显均实测
Apache ManifoldCF 2.31 常用 CLI 命令与工具详解
目标:详解 Apache ManifoldCF 2.31 的常用命令行工具与运维手段——combined/多进程启停脚本、curl 调用 REST API(登录、连接状态、作业生命周期)、脚本引擎 run-script.sh 端到端自动化、密码混淆工具 obfuscate.sh、jps/netstat 进程端口核对与 systemd 托管管理。所有命令与回显均已在 Ubuntu 22.04 + OpenJDK 25 真实环境实测。
一、环境信息
- 操作系统:Ubuntu 22.04 LTS(x86_64,内核 6.8.0-138-generic)
- Java:OpenJDK 25.0.4.1(
openjdk-25-jdk-headless,JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64) - Apache ManifoldCF:2.31,安装目录
/opt/manifoldcf-2.31 - 部署形态:单进程 combined(
example/)与多进程(multiprocess-file-example/)两种均实测 - 端口:8345(Jetty Web + API,两种形态相同)、9001(多进程形态 HSQLDB Server)
- 管理账号:admin / q1w2e3r4(发行包默认 UI 凭据为 admin/admin,API 凭据默认为空)
- 服务托管:systemd 单元
manifoldcf(见 4.7)
二、工具概览
| 工具 / 脚本 | 位置 | 用途 |
|---|---|---|
start-combined.sh / stop-combined.sh | /opt/manifoldcf-2.31/example/ | 单进程 combined 服务启动 / 优雅停止 |
initialize.sh、start-database.sh、start-webapps.sh、start-agents.sh、stop-agents.sh、lock-clean.sh | /opt/manifoldcf-2.31/multiprocess-file-example/ | 多进程形态初始化与数据库 / Web / agents 启停 |
executecommand.sh | /opt/manifoldcf-2.31/multiprocess-file-example/ | 以 agents 类路径执行指定主类 |
run-script.sh | /opt/manifoldcf-2.31/script-engine/ | 执行 .mcf 脚本,端到端创建连接与作业 |
obfuscate.sh | /opt/manifoldcf-2.31/obfuscation-utility/ | 生成密码混淆串 |
curl | 系统自带 | 调用 REST API 管理连接与作业 |
jps | JDK 自带 | 查看 Java 进程与 JVM 参数生效值 |
netstat | net-tools 包 | 校验端口监听 |
systemctl / journalctl | systemd | 服务托管与系统日志 |
三、前置条件
- 已按《Ubuntu 22.04 安装 Apache ManifoldCF 2.31 指南》完成安装,目录为
/opt/manifoldcf-2.31。 - 已安装
net-tools(提供netstat)与curl。 - Java 运行时必须达到 Java 25 级别:2.31 的类文件版本为 69.0(Java 25),用 OpenJDK 11 启动实测直接失败(README.txt 中「Java 7」的描述已过时):
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
bash start-combined.sh
Error: LinkageError occurred while loading main class org.apache.manifoldcf.jettyrunner.ManifoldCFCombinedJettyRunner
java.lang.UnsupportedClassVersionError: org/apache/manifoldcf/jettyrunner/ManifoldCFCombinedJettyRunner has been compiled by a more recent version of the Java Runtime (class file version 69.0), this version of the Java Runtime only recognizes class file versions up to 55.0
本文后续所有命令统一使用:
export JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64
- 发行包内全部
.sh默认无执行权限,解压后先统一补齐执行位(实测触发场景见六、常见问题 Q2):
find /opt/manifoldcf-2.31 -name "*.sh" -exec chmod +x {} \;
四、工具详解
4.1 单进程 combined 服务启停(start-combined.sh / stop-combined.sh)
combined 模式把 UI、authority、API 与 agents 跑在同一个 JVM(ManifoldCFCombinedJettyRunner),Web 上下文为 /mcf。后台启动并把控制台日志重定向到 /opt/mcf-start.log:
cd /opt/manifoldcf-2.31/example
nohup bash start-combined.sh > /opt/mcf-start.log 2>&1 &
tail -5 /opt/mcf-start.log
回显(启动日志尾部):
Successfully registered repository connector 'org.apache.manifoldcf.crawler.connectors.alfrescowebscript.AlfrescoConnector'
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.slack.SlackConnector'
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.rocketchat.RocketChatConnector'
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.email.EmailConnector'
Jetty started.
启动日志头部(节选,先注销旧注册再逐个注册全部内置连接器,注册日志共 70 行):
Configuration file successfully read
Starting jetty...
Configuration file successfully read
Successfully unregistered all domains
Successfully unregistered all output connectors
Successfully unregistered all transformation connectors
Successfully unregistered all mapping connectors
Successfully unregistered all authority connectors
Successfully unregistered all repository connectors
Successfully registered output connector 'org.apache.manifoldcf.agents.output.solr.SolrConnector'
JVM 参数来自 example/combined-options.env.unix(启动脚本逐行读取,前 5 行):
-Xms512m
-Xmx512m
-Dorg.apache.manifoldcf.configfile=./properties.xml
-Dorg.apache.manifoldcf.jettyshutdowntoken=secret_token
-Djava.security.auth.login.config=
启动完成后核对端口与进程:
netstat -tlnp | grep 8345
tcp6 0 0 :::8345 :::* LISTEN 4259/java
jps -v | grep -v Jps
4259 ManifoldCFCombinedJettyRunner -Xms512m -Xmx512m -Dorg.apache.manifoldcf.configfile=./properties.xml -Dorg.apache.manifoldcf.jettyshutdowntoken=secret_token -Djava.security.auth.login.config= --enable-preview
Web 可达性验证(/mcf/ 302 跳转登录页):
curl -s -o /dev/null -w "/mcf/ -> %{http_code}\n" http://localhost:8345/mcf/
curl -s -D- -o /dev/null http://localhost:8345/mcf/ | grep -i location
curl -sL http://localhost:8345/mcf/ | grep title
/mcf/ -> 302
Location: /mcf/login.jsp
<title>Apache ManifoldCF™ Login</title>
UI 登录验证(登录处理页为 /mcf/setupAdminProfile.jsp)。发行包默认 UI 凭据 admin/admin,空凭据被拒:
curl -s -c /tmp/cj3 -o /dev/null -w "HTTP %{http_code} -> %{redirect_url}\n" -d "userID=admin&password=admin" http://localhost:8345/mcf/setupAdminProfile.jsp
curl -s -c /tmp/cj2 -o /dev/null -w "HTTP %{http_code} -> %{redirect_url}\n" -d "userID=&password=" http://localhost:8345/mcf/setupAdminProfile.jsp
HTTP 302 -> http://localhost:8345/mcf/index.jsp
HTTP 302 -> http://localhost:8345/mcf/login.jsp?loginfailed=true
💡 本文环境已把 UI/API 凭据改为 admin/q1w2e3r4(修改方法见《Apache ManifoldCF 2.31 配置参数详解》)。改密并重启进程后再验证:新凭据登录成功、旧默认凭据被拒。
curl -s -c /tmp/cj4 -o /dev/null -w "admin/q1w2e3r4: HTTP %{http_code} -> %{redirect_url}\n" -d "userID=admin&password=q1w2e3r4" http://localhost:8345/mcf/setupAdminProfile.jsp
curl -s -o /dev/null -w "admin/admin: HTTP %{http_code} -> %{redirect_url}\n" -d "userID=admin&password=admin" http://localhost:8345/mcf/setupAdminProfile.jsp
admin/q1w2e3r4: HTTP 302 -> http://localhost:8345/mcf/index.jsp
admin/admin: HTTP 302 -> http://localhost:8345/mcf/login.jsp?loginfailed=true
优雅停止(依赖 combined-options.env.unix 中的 -Dorg.apache.manifoldcf.jettyshutdowntoken 与 jetty.xml 端口,改动后需同步停止脚本,见《Apache ManifoldCF 2.31 配置参数详解》):
bash stop-combined.sh
stop exit: 0
netstat -tlnp | grep 8345 || echo "port 8345 closed"
port 8345 closed
(数秒后 jps 中 ManifoldCFCombinedJettyRunner 消失。)
4.2 多进程形态启停脚本(multiprocess-file-example/)
多进程形态把数据库、Web、agents 拆成独立 JVM 进程,脚本与内部主类的对应关系:
| 脚本 | 内部主类 | 作用 |
|---|---|---|
initialize.sh | org.apache.manifoldcf.crawler.InitializeAndRegister | 初始化数据库并注册全部连接器 |
start-database.sh | org.hsqldb.Server | 启动 HSQLDB Server(独立进程,监听 9001) |
start-webapps.sh | org.apache.manifoldcf.jettyrunner.ManifoldCFJettyRunner | 启动 Web 应用(8345) |
start-agents.sh | org.apache.manifoldcf.agents.AgentRun | 启动 agents 爬取引擎进程 |
stop-agents.sh | org.apache.manifoldcf.agents.AgentStop | 停止 agents |
lock-clean.sh | org.apache.manifoldcf.core.LockClean | 清理锁与同步文件 |
executecommand.sh | (由参数指定) | 以 agents 类路径执行任意主类 |
启动顺序必须严格为:数据库 → 初始化 → Web → agents。
第一步,启动 HSQLDB Server(脚本内部以 "$JAVA_HOME/bin/java" $OPTIONS org.hsqldb.Server -database.0 "file:extdb;hsqldb.tx=mvcc;hsqldb.cache_file_scale=512" -dbname.0 xdb 拉起,与 properties.xml 的 hsqldb protocol/server/instance 参数对应):
cd /opt/manifoldcf-2.31/multiprocess-file-example
nohup bash start-database.sh > /opt/mcf-db.log 2>&1 &
tail -3 /opt/mcf-db.log
[Server@55a561cf]: 2026-10-11 15:18:51.257 HSQLDB server 2.6.1 is online on port 9001
[Server@55a561cf]: To close normally, connect and execute SHUTDOWN SQL
[Server@55a561cf]: From command line, use [Ctrl]+[C] to abort abruptly
netstat -tlnp | grep 9001
tcp6 0 0 :::9001 :::* LISTEN 12036/java
第二步,初始化数据库并注册全部连接器:
bash initialize.sh | tail -4
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.slack.SlackConnector'
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.rocketchat.RocketChatConnector'
Successfully registered notification connector 'org.apache.manifoldcf.crawler.notifications.email.EmailConnector'
Successfully initialized database and registered all connectors
第三步,启动 Web 进程:
nohup bash start-webapps.sh > /opt/mcf-web.log 2>&1 &
tail -1 /opt/mcf-web.log
Jetty started.
netstat -tlnp | grep 8345
tcp6 0 0 :::8345 :::* LISTEN 12227/java
多进程形态的 Web 上下文与 combined 不同:crawler-ui 挂在 /mcf-crawler-ui/,API 服务挂在 /mcf-api-service,authority 服务挂在 /mcf-authority-service:
curl -s -o /dev/null -w "/mcf-crawler-ui/ -> %{http_code}\n" http://localhost:8345/mcf-crawler-ui/
curl -s -o /dev/null -w "/mcf-api-service/json/jobs -> %{http_code}\n" http://localhost:8345/mcf-api-service/json/jobs
curl -s -o /dev/null -w "/mcf-authority-service/ -> %{http_code}\n" http://localhost:8345/mcf-authority-service/
/mcf-crawler-ui/ -> 302
/mcf-api-service/json/jobs -> 200
/mcf-authority-service/ -> 200
(/mcf-crawler-ui/ 的 302 跳转目标为 /mcf-crawler-ui/login.jsp。)
第四步,启动 agents 爬取引擎进程:
nohup bash start-agents.sh > /opt/mcf-agents.log 2>&1 &
tail -3 /opt/mcf-agents.log
Running...
Configuration file successfully read
全部组件进程用 jps -l 核对:
jps -l
12227 org.apache.manifoldcf.jettyrunner.ManifoldCFJettyRunner
12036 org.hsqldb.Server
12294 org.apache.manifoldcf.agents.AgentRun
多进程 UI 登录验证(properties.xml 加入 login 四项属性后,登录处理页为 /mcf-crawler-ui/setupAdminProfile.jsp):
nohup bash start-webapps.sh > /opt/mcf-web2.log 2>&1 &
tail -1 /opt/mcf-web2.log
curl -s -c /tmp/cj5 -o /dev/null -w "login: %{http_code} -> %{redirect_url}\n" -d "userID=admin&password=q1w2e3r4" http://localhost:8345/mcf-crawler-ui/setupAdminProfile.jsp
curl -s -b /tmp/cj5 -o /dev/null -w "index.jsp: %{http_code}\n" http://localhost:8345/mcf-crawler-ui/index.jsp
Jetty started.
login: 302 -> http://localhost:8345/mcf-crawler-ui/index.jsp
index.jsp: 200
停止 agents(发停止信号,进程稍后自行退出):
bash stop-agents.sh
Configuration file successfully read
Shutdown signal sent
tail -3 /opt/mcf-agents.log
Running...
Configuration file successfully read
Shutting down...
(数秒后 jps -l 中 AgentRun 消失。)
清理锁与同步文件:
bash lock-clean.sh
Configuration file successfully read
Synchronization storage cleaned up
停止 Web 进程:不能直接用 agents 类路径的 executecommand.sh 执行关闭主类(实测报类找不到),要用 jetty-options 类路径直接执行 ManifoldCFJettyShutdown:
bash executecommand.sh org.apache.manifoldcf.jettyrunner.ManifoldCFJettyShutdown http://localhost:8345
Error: Could not find or load main class org.apache.manifoldcf.jettyrunner.ManifoldCFJettyShutdown
Caused by: java.lang.ClassNotFoundException: org.apache.manifoldcf.jettyrunner.ManifoldCFJettyShutdown
"$JAVA_HOME/bin/java" $(cat jetty-options.env.unix | tr "\n" " ") org.apache.manifoldcf.jettyrunner.ManifoldCFJettyShutdown http://localhost:8345
exit: 0
(退出码 0,jps 中 ManifoldCFJettyRunner 消失。)
4.3 进程与端口核对(jps / netstat)
jps -v 直接显示各进程生效的 JVM 参数(多进程形态实测,末尾未补全处为采集时截断,回显按实际保留):
jps -v | grep -v Jps
12576 AgentRun -Xms512m -Xmx512m -Dorg.apache.manifoldcf.configfile=./properties.xml -Djava.security.auth.login.config= --enable-preview -Dorg.apache.
12227 ManifoldCFJettyRunner -Xms256m -Xmx256m -Dorg.apache.manifoldcf.configfile=./properties.xml -Dorg.apache.manifoldcf.jettyshutdowntoken=secret_to
12036 Server -Xms256m -Xmx256m --enable-preview
各 options 文件与堆默认值对照(2.31 发行包):
| options 文件 | 所属进程 | 默认堆 |
|---|---|---|
| example/combined-options.env.unix | combined 单进程 | -Xms512m -Xmx512m |
| multiprocess-file-example/jetty-options.env.unix | 多进程 Web | -Xms256m -Xmx256m |
| multiprocess-file-example/options.env.unix | 多进程 agents | -Xms512m -Xmx512m |
| multiprocess-file-example/hsqldb-options.env.unix | HSQLDB Server | -Xms256m -Xmx256m |
| script-engine/options.env.unix | 脚本引擎 | -Xms8m -Xmx8m(跑示例需调大,见 4.5) |
端口速查:
| 端口 | 进程 | 说明 |
|---|---|---|
| 8345 | ManifoldCFCombinedJettyRunner / ManifoldCFJettyRunner | Web UI + API(combined 与多进程同端口) |
| 9001 | org.hsqldb.Server | 多进程形态 HSQLDB Server |
4.4 REST API 运维(curl)
API 基址:combined 模式为 /mcf/api/json/,多进程模式为 /mcf-api-service/json/;资源名为小写复数。
⚠️ 未带登录会话时 GET 只返回空对象,容易误判为「无数据」,必须先 LOGIN 再携带 cookie:
curl -s http://localhost:8345/mcf/api/json/jobs
{}
登录(请求体 JSON 对应 properties.xml 的 org.apache.manifoldcf.apilogin.name / apilogin.password):
rm -f /tmp/apicj2
curl -s -c /tmp/apicj2 -X POST -H "Content-Type: application/json" -d "{\"userID\":\"admin\",\"password\":\"q1w2e3r4\"}" http://localhost:8345/mcf/api/json/LOGIN
{}
之后带 cookie 查询(-b /tmp/apicj2)。为便于复用,把 curl 前缀与作业号存入 shell 变量(实际作业号以 jobs 查询结果为准):
C="curl -s -b /tmp/apicj2"; JID=1791731450272
作业列表:
$C http://localhost:8345/mcf/api/json/jobs
{"job":{"_children_":[{"_type_":"id","_value_":"1791731450272"},{"_type_":"description","_value_":"File system crawl of \/opt\/manifoldcf-2.31\/doc"},{"_type_":"repository_connection","_value_":"File System"},{"_type_":"document_specification","startpoint":{"include":[{"_attribute_match":"*","_value_":"","_attribute_type":"file"},{"_attribute_match":"*","_value_":"","_attribute_type":"directory"}],"_attribute_path":"\/opt\/manifoldcf-2.31\/doc","_value_":""}},{"_type_":"pipelinestage","_children_":[{"_type_":"stage_id","_value_":"0"},{"_type_":"stage_isoutput","_value_":"true"},{"_type_":"stage_connectionname","_value_":"Null Output"},{"_type_":"stage_specification"}]},{"_type_":"start_mode","_value_":"manual"},{"_type_":"run_mode","_value_":"scan once"},{"_type_":"hopcount_mode","_value_":"never delete"},{"_type_":"priority","_value_":"5"},{"_type_":"recrawl_interval","_value_":"86400000"},{"_type_":"max_recrawl_interval","_value_":"infinite"},{"_type_":"expiration_interval","_value_":"infinite"},{"_type_":"reseed_interval","_value_":"3600000"}]}}
单个作业(长回显截取前 400 字符):
$C http://localhost:8345/mcf/api/json/jobs/1791731450272 | head -c 400
{"job":{"_children_":[{"_type_":"id","_value_":"1791731450272"},{"_type_":"description","_value_":"File system crawl of \/opt\/manifoldcf-2.31\/doc"},{"_type_":"repository_connection","_value_":"File System"},{"_type_":"document_specification","startpoint":{"include":[{"_attribute_match":"*","_value_":"","_attribute_type":"file"},{"_attribute_match":"*","_value_":"","_attribute_type":"directory"}]
作业状态(含队列 / 处理文档计数):
$C http://localhost:8345/mcf/api/json/jobstatuses
{"jobstatus":{"_children_":[{"_type_":"job_id","_value_":"1791731450272"},{"_type_":"status","_value_":"done"},{"_type_":"start_time","_value_":"1791731452861"},{"_type_":"end_time","_value_":"1791731492872"},{"_type_":"documents_in_queue","_value_":"3714"},{"_type_":"documents_outstanding","_value_":"0"},{"_type_":"documents_processed","_value_":"3714"},{"_type_":"queue_exact","_value_":"true"},{"_type_":"outstanding_exact","_value_":"true"},{"_type_":"processed_exact","_value_":"true"}]}}
连接状态检查(URL 中的连接名需 URL 编码,如 File%20System):
$C "http://localhost:8345/mcf/api/json/status/repositoryconnections/File%20System"
$C "http://localhost:8345/mcf/api/json/status/outputconnections/Null%20Output"
{"check_result":"Connection working"}
{"check_result":"Connection working"}
作业生命周期操作(均为 PUT,动作是否成功取决于作业当前状态)。done 状态下 restart / pause / resume 均被拒绝:
$C -X PUT http://localhost:8345/mcf/api/json/restart/$JID
{"error":"Job 1791731450272 is not restartable"}
$C -X PUT http://localhost:8345/mcf/api/json/pause/$JID
{"error":"Job 1791731450272 is not active"}
$C -X PUT http://localhost:8345/mcf/api/json/resume/$JID
{"error":"Job 1791731450272 is not paused"}
done 状态的 scan once 作业重新启动:
$C -X PUT http://localhost:8345/mcf/api/json/start/$JID
{}
(10 秒后查询 jobstatuses/$JID,status 字段变为 running。)
运行中作业暂停:
$C -X PUT http://localhost:8345/mcf/api/json/pause/$JID
{}
(数秒后 status 依次经 stopping 进入 paused。)
恢复运行:
$C -X PUT http://localhost:8345/mcf/api/json/resume/$JID
{}
(status 先变为 resuming,随后回到 running。)
中止作业:
$C -X PUT http://localhost:8345/mcf/api/json/abort/$JID
{}
(scan once 作业很快收尾,status 经 notifying 回到 done。)
REST 资源速查(2.31 官方 programmatic-operation.html 资源表关键行,以 combined 基址为例;standalone 模式把 /mcf/api/json/ 换成 /mcf-api-service/json/):
| 资源 / 端点 | 方法 | 说明 |
|---|---|---|
| LOGIN | POST | 登录,请求体 {"userID":...,"password":...} |
| outputconnections | GET | 输出连接列表 |
| outputconnections/<连接名> | GET / PUT / DELETE | 单个输出连接读 / 写 / 删 |
| status/outputconnections/<连接名> | GET | 输出连接状态检查 |
| repositoryconnections | GET | 仓库连接列表 |
| status/repositoryconnections/<连接名> | GET | 仓库连接状态检查 |
| transformationconnections、mappingconnections、authorityconnections、notificationconnections | GET | 转换 / 映射 / 权限 / 通知连接,形态同上 |
| authoritygroups | GET | 权限组列表 |
| jobs | GET / POST | 作业列表 / 创建作业(POST 返回 {"job_id":...}) |
| jobs/<作业 ID> | GET / PUT / DELETE | 单个作业读 / 写 / 删 |
| jobstatuses、jobstatusesnocounts | GET | 作业状态(含 / 不含文档计数) |
| start/<作业 ID>、abort/<作业 ID>、restart/<作业 ID>、pause/<作业 ID>、resume/<作业 ID> | PUT | 生命周期操作(均有实测回显) |
| startminimal/<作业 ID>、restartminimal/<作业 ID>、reseed/<作业 ID> | PUT | ⚠️ 官方文档定义,本次未逐个实测 |
| repositoryconnectionjobs/<连接名>、repositoryconnectionhistory/<连接名> | GET | ⚠️ 官方文档定义,本次未逐个实测 |
⚠️ 上表端点路径与官方 2.31 文档一致;标 ⚠️ 的两行未在实测中逐一调用,使用前以官方 programmatic-operation.html 为准。
资源名大小写敏感、必须小写复数:jobs、outputconnections、repositoryconnections、authorityconnections、authoritygroups、transformationconnections、notificationconnections 实测均返回 200;写错大小写或用错资源名报 404:
curl -s -w "\nHTTP %{http_code}\n" http://localhost:8345/mcf/api/json/Jobs
{"error":"Unrecognized resource."}
HTTP 404
API 协议段写错(api 与 json 之间插入其他段)报 400:
curl -s http://localhost:8345/mcf/api/service/json/Jobs
Error 400 Unknown API protocol: service
4.5 脚本引擎自动化(run-script.sh)
script-engine/ 目录提供 run-script.sh,执行 .mcf 脚本即可完成「登录 → 建连接 → 建作业 → 等待完成」的端到端流程,发行包自带文件系统爬取示例 file-crawl-example.mcf。参数顺序:run-script.sh <脚本文件> <爬取路径> <API基址> <用户名> <密码>,API 基址必须带尾部斜杠。
⚠️ 默认 JVM 堆只有 8MB(script-engine/options.env.unix 前两行为 -Xms8m、-Xmx8m),直接跑示例会 OOM:
bash run-script.sh file-crawl-example.mcf /opt/manifoldcf-2.31/doc http://localhost:8345/mcf/api/ admin q1w2e3r4
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.base/java.lang.StringUTF16.compress(StringUTF16.java:212)
at java.base/java.lang.String.<init>(String.java:4842)
at java.base/java.lang.String.<init>(String.java:314)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:366)
at java.base/java.io.BufferedReader.readLine(BufferedReader.java:400)
at org.apache.http.conn.util.PublicSuffixListParser.parseByType(PublicSuffixListParser.java:111)
at org.apache.http.conn.util.PublicSuffixMatcherLoader.load(PublicSuffixMatcherLoader.java:54)
...
at org.apache.manifoldcf.scriptengine.ScriptParser.getHttpClient(ScriptParser.java:1224)
at org.apache.manifoldcf.scriptengine.POSTCommand.parseAndExecute(POSTCommand.java:71)
解决:编辑 script-engine/options.env.unix,把前两行改为 -Xms64m、-Xmx256m,重新执行:
bash run-script.sh file-crawl-example.mcf /opt/manifoldcf-2.31/doc http://localhost:8345/mcf/api/ admin q1w2e3r4
Login successful
Output connection created (or already exists)
Repository connection created (or already exists)
Job created
The job id is 1791731450272
The job completed
脚本创建的作业可用 4.4 的 jobstatuses 查询:本次实测 start_time 与 end_time 相差约 40 秒,处理 3714 个文档。
4.6 密码混淆工具(obfuscate.sh)
obfuscation-utility/ 目录的 obfuscate.sh 生成可写入 properties.xml 的密码混淆串:
cd /opt/manifoldcf-2.31/obfuscation-utility && bash obfuscate.sh q1w2e3r4
m/s6WoTCdVjAr7/FCSYBnv8o3zcVzVixwjGP/+rgAeA=
混淆串的用法:写入「原属性名 + .obfuscated 后缀」的属性(原明文属性删除或保留均可,.obfuscated 优先):
<property name="org.apache.manifoldcf.login.password.obfuscated" value="m/s6WoTCdVjAr7/FCSYBnv8o3zcVzVixwjGP/+rgAeA="/>
实测:按上述写法配置并重启后,admin/q1w2e3r4 登录成功(302 → index.jsp);而把裸混淆串或带 {Obfuscated} 前缀的值直接写在原属性名 org.apache.manifoldcf.login.password 上,两种写法登录均失败(跳转 login.jsp?loginfailed=true)。
4.7 systemd 托管管理(systemctl / journalctl)
生产服务器建议用 systemd 托管 combined 进程。单元文件 /etc/systemd/system/manifoldcf.service:
[Unit]
Description=Apache ManifoldCF 2.31 (combined example)
After=network.target
[Service]
Type=simple
User=root
Environment=JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64
WorkingDirectory=/opt/manifoldcf-2.31/example
ExecStart=/opt/manifoldcf-2.31/example/start-combined.sh
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
启用并启动(每条命令单独执行):
systemctl daemon-reload
systemctl enable manifoldcf
Created symlink /etc/systemd/system/multi-user.target.wants/manifoldcf.service → /etc/systemd/system/manifoldcf.service.
systemctl start manifoldcf
systemctl is-active manifoldcf
active
状态详情(head -12 回显节选至 CPU 行,其后 CGroup / 进程树行从略;java 进程命令行用 jps -v 核对):
systemctl status manifoldcf --no-pager | head -12
● manifoldcf.service - Apache ManifoldCF 2.31 (combined example)
Loaded: loaded (/etc/systemd/system/manifoldcf.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2026-10-11 15:29:26 UTC; 30s ago
Main PID: 13917 (start-combined.)
Tasks: 63 (limit: 5753)
Memory: 510.9M
CPU: 17.247s
服务日志用 journalctl 过滤(回显节选):
journalctl -u manifoldcf --no-pager | grep -E "Started|Starting"
Oct 11 15:29:26 70fbfb7e3e4b systemd[1]: Started Apache ManifoldCF 2.31 (combined example).
Oct 11 15:29:27 70fbfb7e3e4b start-combined.sh[13919]: Starting jetty...
Oct 11 15:29:32 70fbfb7e3e4b start-combined.sh[13919]: Successfully registered repository connector 'org.apache.manifoldcf.crawler.connectors.dropbox.DropboxRepositoryConnector'
重启与停止:
systemctl restart manifoldcf && sleep 25 && systemctl is-active manifoldcf
active
systemctl stop manifoldcf && sleep 8 && systemctl is-active manifoldcf
inactive
(停止后 jps 无任何 java 进程。)
💡 systemd 托管时
WorkingDirectory必须是example/(多进程形态为multiprocess-file-example/),否则启动脚本读不到相对路径的 properties.xml、jetty.xml,日志中不会出现Configuration file successfully read。
4.8 日志定位与查看
- combined 运行日志:
/opt/manifoldcf-2.31/example/logs/manifoldcf.log(由同目录 logging.xml 的 file appender 定义,输出路径logs/manifoldcf.log相对工作目录)。 - 手工启动时 nohup 重定向的控制台日志:
/opt/mcf-start.log(combined)、/opt/mcf-db.log(HSQLDB)、/opt/mcf-web.log(Web)、/opt/mcf-agents.log(agents)。 - systemd 部署时控制台输出进 journal:
journalctl -u manifoldcf --no-pager。
运行日志中可核对 API 操作报错(4.4 中对 done 状态作业做 restart / pause / resume 时的异常记录):
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Job 1791731450272 is not active
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Job 1791731450272 is not paused
org.apache.manifoldcf.core.interfaces.ManifoldCFException: Job 1791731450272 is not restartable
⚠️ 以下为通用查看命令(证据采集时未单独记录该命令回显,输出以实际日志内容为准):
tail -n 50 /opt/manifoldcf-2.31/example/logs/manifoldcf.log
五、注意事项
- ManifoldCF 2.31 的类文件为 Java 25 级别(class file 69.0),Java 11 及以下无法启动;README.txt 中「Java 7」的描述已过时。
- 发行包内全部
.sh默认无执行权限,解压后先统一chmod +x,或始终以bash前缀执行。 - combined 与多进程的 Web 上下文与 API 基址不同:
/mcf/、/mcf/api/json/对应/mcf-crawler-ui/、/mcf-api-service/json/,访问错路径会 404。 - REST 资源名小写复数、大小写敏感;未带登录会话时 GET 返回空
{},不要误判为「无数据」。 - 作业生命周期动作有状态前提:restart 需可重启状态、pause 需 active、resume 需 paused,否则返回 error JSON。
- 多进程形态停止 Web 必须用 jetty-options 类路径执行
ManifoldCFJettyShutdown,agents 类路径会 ClassNotFoundException。 run-script.sh的 API 基址必须带尾部斜杠;脚本引擎默认堆 8MB 过小,先调大再跑示例。- 多条命令逐个单独执行,避免对运行中的服务做批量循环操作。
六、常见问题(FAQ)
Q1:启动报 UnsupportedClassVersionError(class file version 69.0)?
JDK 版本过低。切换到 OpenJDK 25(export JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64)后重新启动;完整错误回显见三、前置条件。
Q2:执行脚本报 ./executecommand.sh: Permission denied?
发行包 .sh 未带执行位。实测 initialize.sh 触发:
initialize.sh: line 21: ./executecommand.sh: Permission denied
执行 find /opt/manifoldcf-2.31 -name "*.sh" -exec chmod +x {} \; 补齐权限后重试。
Q3:initialize.sh 报数据库连接被拒绝?
未先启动数据库。先用 start-database.sh 启动 HSQLDB Server(9001 端口在线日志见 4.2),再执行 initialize.sh。实测异常栈尾部特征:
at java.base/java.net.Socket.connect(Socket.java:659)
at java.base/java.net.Socket.connect(Socket.java:597)
at org.hsqldb.server.HsqlSocketFactory.createSocket(Unknown Source)
Q4:combined 模式访问 /mcf-crawler-ui/ 返回 404?
combined 模式只有 /mcf 一个上下文,/mcf-crawler-ui/、/mcf-api-service 路径实测 404 Not Found;改用 http://localhost:8345/mcf/。/mcf-crawler-ui/ 仅多进程形态使用。
Q5:API 报 Unknown API protocol 或 Unrecognized resource?
协议段写错返回 Error 400 Unknown API protocol: service;资源名大小写 / 拼写错误返回 404 {"error":"Unrecognized resource."}。正确基址与资源名见 4.4。
Q6:run-script.sh 抛 OutOfMemoryError?
脚本引擎默认堆 -Xms8m / -Xmx8m 太小。把 script-engine/options.env.unix 前两行改为 -Xms64m、-Xmx256m 后重跑;完整错误栈与成功回显见 4.5。
Q7:PUT restart / pause / resume 返回 {"error":"Job ... is not ..."}?
作业状态不满足动作前提(done 状态不能 restart / pause / resume)。先 GET jobstatuses 确认状态:running 可 pause / abort,paused 可 resume;scan once 作业完成后再跑用 start。实测错误回显与状态流转见 4.4。
Q8:stop-agents.sh 执行后 agents 没立刻停?
停止是信号式的:日志出现 Shutdown signal sent 表示信号已发出,/opt/mcf-agents.log 随后出现 Shutting down...,稍等数秒再用 jps -l 确认 AgentRun 是否消失(回显见 4.2)。
七、参考链接
- 2.31 最终用户文档:https://manifoldcf.apache.org/release/release-2.31/en_US/end-user-documentation.html
- 2.31 programmatic-operation(REST API 资源表):https://manifoldcf.apache.org/release/release-2.31/en_US/programmatic-operation.html
- 2.31 构建与部署文档:https://manifoldcf.apache.org/release/release-2.31/en_US/how-to-build-and-deploy.html
- 项目主页:https://manifoldcf.apache.org/