Apache Pulsar in WSL: Standalone Topics, Subscriptions, and Storage

Apache Pulsar’s standalone mode is useful for exercising topic naming, client configuration, schemas, subscriptions, acknowledgements, and application error paths from a WSL development environment. The Pulsar 5.0 documentation describes standalone as running the components of a cluster within a single JVM process. That makes startup convenient, but it does not reproduce independent brokers, bookies, metadata services, failure domains, or production recovery. Treat it as a protocol and application integration lab.

Pulsar’s service and storage paths should stay inside the WSL Linux filesystem for a Linux process. Keep the distribution archive, data, logs, and any application fixtures under $HOME or another Linux directory. Microsoft’s WSL filesystem documentation explains why Linux workloads generally perform better within the distro filesystem than under /mnt/c. If you need to test Windows-side file ingestion, use a separate disposable scenario rather than mixing it into the broker baseline.

Match the server, client, and Java versions

Pulsar server and client requirements are release-specific. The 5.0.x standalone guide currently requires a 64-bit Java 21 or later runtime for the server; the client library can have a separate minimum. Verify the exact requirement for the archive and SDK you use, and record all versions. Do not infer the broker runtime requirement from the client runtime or an older Pulsar article.

Download and extract the official binary distribution under a versioned path. Start standalone in a foreground terminal first:

cd "$HOME/pulsar-lab/apache-pulsar"
java -version
bin/pulsar standalone

The standalone guide explains that the process creates local data and log directories. Keep the terminal visible and use a second WSL terminal for topic and client commands. The default public/default namespace is described as development-oriented. Create a clearly named lab topic rather than reusing a production topic name or pointing test code at a shared cluster.

Produce and consume a bounded test stream

Pulsar topics are addressed through a tenant, namespace, and topic hierarchy. For a small local test, create one persistent topic under the documented development namespace, then send a few uniquely identified messages with the CLI or the language SDK your application uses. Consume them through an explicitly named subscription and verify the message body, key, schema, and acknowledgement behavior. Keep a copy of the exact commands or program inputs so a second run is comparable.

Test the difference between a new subscription and an existing subscription. An exclusive, failover, shared, or key-shared consumer model has different dispatch behavior; check the concepts guide for the selected client and topic type. Do not infer global exactly-once application effects from a broker acknowledgement. Your consumer may process a message and fail before acknowledging it, so application side effects should be idempotent or protected by an application-level deduplication key.

Use a bounded backlog. Publish a fixed number of test messages, stop the consumer at a controlled point, inspect backlog, restart it, and verify which messages are redelivered or acknowledged. This tests a local behavior under a known state. It does not test loss of a broker/bookie host or recovery from a metadata-store outage because standalone consolidates the services into one local failure domain.

Understand standalone storage and state ownership

Pulsar standalone embeds BookKeeper storage and a metadata implementation suitable for local development. The 5.0.x guide describes RocksDB as the default metadata store and identifies directories created for data and logs. Do not move these directories while the broker is active. Do not delete them to “reset” a failed client before confirming the server is stopped and the path belongs to this lab.

Standalone state can survive process restarts depending on the exact configuration and version, but that does not make it a production backup. Before reusing local data with another release, follow the standalone upgrade guidance and back up data, metadata, and configuration together. Keep the original binary or container image available for a rollback test. A copy of a live WSL VHDX is not an application-consistent Pulsar backup unless the server is quiesced or the documented backup procedure is used.

Use the lab to validate that a topic, schema, and subscription are created as expected. Use the Admin CLI to inspect topic stats and backlog rather than guessing from producer output. Keep any test tenant, namespace, and topic names unique to the lab; cleanup should target only those names. Avoid broad deletion commands copied from an example cluster.

Set message keys deliberately when ordering is part of the contract. Producer sequence and consumer observation order can depend on partitioning and subscription type. A local standalone topic can validate client serialization and key handling, but it does not validate ordering across a partitioned topic under broker reassignment or cluster failure. Include messages with the same key and different keys in a test fixture and assert only the ordering the application actually requires.

Schema compatibility should be tested as a contract between producer and consumer. Create one schema version, publish a small message, and verify the consumer decodes it using the intended SDK. Then test the proposed evolution rule with an additive or otherwise approved field change. Do not assume a JSON payload is schema-governed merely because it is valid JSON. Schema validation, topic policy, and client serialization must all be configured for the behavior under test.

Keep consumer progress visible during the exercise. Acknowledgement, negative acknowledgement, redelivery, and subscription cursor state explain why messages may reappear or remain in backlog. Capture the starting cursor and stop point so a later run can distinguish expected replay from duplicate publication. Do not delete a subscription before collecting the evidence needed to understand its backlog behavior.

Subscription type changes the consumer contract. Use an exclusive subscription when one consumer owns a test stream, and choose shared or failover behavior only when the application is explicitly testing those semantics. A producer send acknowledgment, consumer receipt, and application-level processing acknowledgment are different events. To test redelivery, deliberately stop or negatively acknowledge a disposable message and observe the configured retry behavior; do not infer exactly-once business effects from a broker acknowledgment. Make the test message carry a stable ID so duplicate delivery can be detected by the consumer fixture.

Keep listeners local and separate from a production security model

The Pulsar documentation states that a cluster is not intended to be exposed to the public internet and expects a network perimeter boundary. Local standalone mode is for development and testing. Keep its advertised address and listener scope appropriate to the local clients. If a Windows client cannot reach the service, verify the WSL network mode and listener first. Do not widen the endpoint to every interface as an unexamined workaround.

WSL NAT localhost forwarding and mirrored networking have different behaviors. Test Linux-to-Linux and Windows-to-Linux paths independently, and distinguish the broker’s service ports from BookKeeper or metadata traffic. A Windows app connecting to one port does not show that all Pulsar components are reachable in a distributed deployment. Do not expose admin APIs or unencrypted development endpoints to an untrusted network.

Troubleshooting sequence

If standalone fails before readiness, inspect Java version, archive integrity, disk space, port conflicts, and the server logs. If a producer connects but the consumer sees nothing, compare the full topic name, subscription name, schema, and active namespace. If a consumer receives duplicates, inspect acknowledgement timing, negative acknowledgements, redelivery settings, and whether application code committed an external side effect before acking. If backlog does not shrink, confirm which subscription you are observing and whether a consumer is active.

If the WSL VHDX grows, identify broker data, logs, application fixtures, and downloaded archives separately. Retention and backlog policies should be deliberately bounded for a workstation. Do not benchmark sustained throughput from a tiny example or compare standalone latency to a multi-node Pulsar deployment without matching JVM, storage, message size, and topology.

Acceptance criteria

Accept the lab when the Pulsar release and Java versions are recorded, standalone starts with an explicit Linux data path, a test topic and subscription can be inspected, a bounded producer/consumer exercise returns expected messages, and shutdown/restart behavior is understood. Keep the test isolated from production credentials and topics.

Pulsar standalone in WSL is useful for client compatibility and messaging semantics. It does not validate multi-broker replication, BookKeeper failure handling, geo-replication, production security, or capacity.

Related:

Sources: