结论先说清楚:CardputerZero 不适合作为本地大语言模型主机。可以做极小规模的量化实验,但不能指望它在本地承担一个真正能调用工具的现代 agent。

内存账很紧

官方规格是 512 MB LPDDR2。这 512 MB 要同时放下 Linux 内核与系统服务、官方图形启动器(launcher 使用 LVGL,跑在 framebuffer 上)以及可能打开的终端或 ZClaw,还要再放下模型权重、推理运行时的计算缓冲区,以及上下文的 KV 缓存。

启动图形界面之后,空闲内存会明显少于 512 MB。任何「模型文件只有四百多 MB,所以能跑」的算法都不可靠:文件体积不等于运行时占用,短上下文和长上下文的差别也很大。动手之前,应先 SSH 到设备上执行 free -h,以当时的可用内存为准。

同级硬件上被核对过的,只是极小模型

CardputerZero 的 SoC 与 Raspberry Pi Zero 2 W 同属 RP3A0 / BCM2837、四核 Cortex-A53 @ 1 GHz、512 MB 这一级。社区有人在 Zero 2 W 上用 llama.cpp 运行 SmolLM2-135M-Instruct 的 Q4_K_M GGUF,并报告可以在很短的上下文(例如 256 tokens)下工作;有的写法还依赖 swap。这些是可核对的社区记录,针对的是树莓派 Zero 2 W,通常还是精简或无桌面的系统,不是 CardputerZero 官方镜像上的实测。

即便这类约 1 亿参数的小模型能在内存里站住,它们的能力也远达不到可靠的工具调用 agent:上下文短、幻觉多、指令遵循不稳定。把它们当实验可以,当日常助手不行。

更大的模型更不乐观。例如有记录在同样 512 MB、但更慢的 32 位开发板上尝试过 TinyLlama 1.1B 的极低比特量化,加载就要数分钟,生成速度可以低到几乎无法对话。这类结果说明:1B 级模型即使「勉强塞进去」,也不实用。本文不会给出一套声称已在 CardputerZero 官方系统上跑通的本地大模型命令。

至于只有一千多万参数、用来写小故事的模型(例如 TinyStories-15M 一类),它们能证明「设备可以做点推理」,但几乎没有 agent 价值。

不要用 swap 假装内存变大

在 microSD 上建立很大的 swap,有时能让进程不立刻被 OOM 杀掉,但会严重拖慢推理,并增加闪存磨损。官方镜像以 microSD 为唯一存储,不值得为一次演示去磨损存储卡。

若只是想观察极小模型能否启动,更负责任的做法是:先看 free -h;在另一台计算机上交叉编译 llama.cpp(官方镜像是 aarch64 Debian / Raspberry Pi OS 一类系统);只试验明显小于当前可用内存的模型,并使用很短的上下文。社区有报告称,直接在 Zero 2 W 上编译 llama.cpp 可能需要大约一整天。那不是合理的设备日常用法,也不是已经在 CardputerZero 上核实过的步骤。

和官方「便携式 AI」表述怎么对齐

官方把「便携式 AI / 视觉原型验证」列为典型场景。更合理的理解是:这台机器便于带到现场,接上相机或传感器,把数据交给远端模型,或做非常轻量的边缘处理——而不是在 512 MB 里部署一套本地聊天模型。视觉原型同样受内存和算力限制;需要较重的视觉模型时,仍然应该把推理放到云端或更强的电脑上,设备只负责采集。

建议

日常 agent 走云端或局域网远端 API,见 云端工具调用。本地只考虑远小于可用内存的量化小模型,并接受短上下文和不稳定的质量。不要把原版 ESP32 Cardputer 上的 MCU 侧演示,当成这台 Linux 电脑的本地 LLM 方案。

The conclusion up front: CardputerZero is not a suitable host for a local large language model. Tiny quantized experiments may be possible. A modern tool-calling agent that actually runs on the device is not a realistic daily setup.

The memory budget is tight

Official specifications list 512 MB of LPDDR2. That single pool has to hold the Linux kernel and system services, the official graphical launcher (LVGL on a framebuffer), and whatever terminal or ZClaw session is open—plus model weights, the inference runtime’s compute buffers, and the context KV cache.

After the graphical UI starts, free memory is substantially less than 512 MB. “The file is only a little over 400 MB, so it will fit” is not a sound test: file size is not runtime size, and a short context is very different from a long one. SSH in and run free -h before talking about models, and trust the number you see on that machine.

What has been checked on similar hardware is only a tiny model

CardputerZero’s SoC is in the same class as a Raspberry Pi Zero 2 W: RP3A0 / BCM2837, quad-core Cortex-A53 at 1 GHz, 512 MB. Community write-ups describe running a SmolLM2-135M-Instruct Q4_K_M GGUF with llama.cpp on a Zero 2 W, often with a very short context (for example 256 tokens), and sometimes with swap. Those are verifiable community records about a Pi Zero 2 W, usually on a slim or headless image. They are not measured results on the official CardputerZero image.

Even if a model of about 135 million parameters stays resident, it is far from a reliable tool-calling agent: the context is short, hallucinations are common, and instruction following is uneven. That is a reasonable experiment, not a daily assistant.

Larger models look worse. There are reports of forcing a highly quantized TinyLlama 1.1B onto other 512 MB boards, including slower 32-bit boards, with multi-minute load times and generation so slow that a conversation barely works. Those results are a warning, not a recipe: a 1B-class model that merely “fits” is still impractical. This article will not publish a command sequence that claims to have been verified on official CardputerZero software.

Still smaller toys, such as a 15-million-parameter TinyStories-class model, can show that the hardware can do some inference. They have almost no value as an agent.

Do not treat swap as extra RAM

A large swap file on microSD can sometimes keep a process from being killed immediately, but it makes inference much slower and wears the flash. The official storage path is microSD; burning a card for a one-off demo is a poor trade.

If you only want to see whether a tiny model will start, a more responsible approach is: read free -h; cross-compile llama.cpp on a stronger computer (the official image is an aarch64 Debian / Raspberry Pi OS-class system); try only a model that is clearly smaller than the free RAM you measured; and keep the context short. Community reports say compiling llama.cpp directly on a Zero 2 W can take on the order of a day. That is not a reasonable daily workflow, and it is not a procedure verified on CardputerZero itself.

How this sits with the official “portable AI” wording

Official documentation lists “portable AI / vision prototyping” as a typical use. The reading that fits the hardware is: take the computer into the field, attach a camera or a sensor, and send the data to a remote model—or do only very light edge processing. It is not a claim that a local chat model will live comfortably in 512 MB. Vision prototypes face the same memory and compute limits. If the vision model is heavy, keep inference in the cloud or on a stronger computer and let the device capture.

Practical advice

For a daily agent, use a cloud or LAN API; see cloud tool-calling. Locally, consider only a quantized model far smaller than the free RAM you measured, and accept short context and uneven quality. Do not treat MCU-side demos from the original ESP32 Cardputer as a local LLM method for this Linux computer.