Using ZFS as the Storage Control Plane

Based on a LinkedIn post originally published on 28 January 2026

On HG000009, the front-end data store node in my home lab, ZFS serves as the central storage control plane.

I use the term control plane deliberately.

ZFS is not merely the filesystem placed on top of a disk. It is the layer through which back-end storage resources are aggregated, organised, protected, observed and presented to consumers.

The physical and remotely provided storage devices become part of ZFS-managed pools. ZFS then provides the storage from which client-facing services can be created, including:

• iSCSI targets;

• SMB shares where appropriate;

• datasets;

• volumes;

• snapshots;

• and replicated storage states.

This design makes storage behaviour explicit.

Instead of allowing each application or client to interact directly with individual devices, HG000009 provides a controlled point at which data-integrity, allocation, recovery and presentation decisions are made.

What “control plane” means here

In large distributed platforms, the term control plane often refers to components that define and coordinate the desired state of other systems.

My use of the term for ZFS is architectural rather than an assertion that ZFS is a separate distributed controller.

Within HG000009, ZFS is the authoritative layer responsible for questions such as:

• Which devices belong to which storage pool?

• How is redundancy organised?

• Which datasets and volumes exist?

• Which properties apply to them?

• How is capacity allocated?

• Which snapshots are retained?

• What is replicated?

• How is storage health observed?

• How is a degraded state handled?

• Which storage units are presented to consumers?

These decisions are centralised in one inspectable storage model.

This is preferable to allowing them to emerge implicitly from unrelated mount points, scripts, device paths and application-specific configurations.

Storage abstraction with visible dependencies

Abstraction is useful because consumers should not need to understand every physical detail.

An SMB client should not need to know which disk enclosure contains a particular block. A system consuming an iSCSI target should not have to manage the individual devices from which that target is constructed.

However, abstraction must not conceal dependencies from operators.

A logical volume may depend on:

• a ZFS pool;

• one or more virtual devices;

• back-end iSCSI sessions;

• network interfaces;

• remote storage nodes;

• physical enclosures;

• and individual disks.

Those relationships must remain documented and observable.

A storage abstraction becomes dangerous when it makes the service appear independent of the components that can cause it to fail.

The purpose of the control plane is therefore not to hide reality. It is to provide a stable interface while preserving operational understanding.

End-to-end data integrity

One of the principal reasons for using ZFS is its data-integrity model.

Traditional storage stacks may rely on several independent layers:

• physical devices;

• RAID controllers;

• logical-volume managers;

• filesystems;

• caching;

• and application-level validation.

Each layer may assume that the data received from the layer below is correct.

ZFS uses checksums to detect whether stored data still matches what was originally written. The checksum is stored separately from the data block it protects, reducing the risk that corruption of one location makes both the data and its validation information appear consistent.

When a block is read, ZFS can verify its contents.

This provides protection against forms of silent corruption that might otherwise remain undetected until the affected information is needed.

Detection, however, is not the same as correction.

ZFS can repair damaged data automatically only when another valid copy exists through an appropriate redundancy configuration. Without redundancy, it may detect corruption but have no source from which to reconstruct the correct block.

Data integrity therefore depends on both the capabilities of ZFS and the architecture of the pool.

Copy-on-write behaviour

ZFS uses copy-on-write semantics.

When existing data changes, the new information is written to a new location rather than immediately overwriting the original block in place. Once the new data is safely written, the relevant metadata is updated to reference it.

This design supports several useful properties:

• consistent on-disk state;

• efficient snapshots;

• reduced dependence on traditional filesystem-repair procedures;

• and clearer recovery behaviour after interruption.

Copy-on-write does not make storage immune to every failure.

Hardware faults, incorrect pool design, exhausted capacity, operator mistakes and simultaneous device failures can still cause serious problems.

It does provide a more controlled model for updating stored data.

Datasets as operational boundaries

A ZFS pool does not need to be treated as one large undifferentiated filesystem.

Datasets allow storage to be divided according to operational purpose.

Different datasets may have their own:

• mount points;

• quotas;

• reservations;

• compression settings;

• snapshot policies;

• replication rules;

• access permissions;

• and other properties.

This makes datasets useful as management boundaries.

For example, data with different retention requirements should not automatically share the same snapshot policy. A workload requiring a strict capacity guarantee may need a reservation. Another may require a quota to prevent it from consuming the complete pool.

The objective is to make policy explicit at the appropriate level.

Zvols and iSCSI targets

ZFS volumes, commonly called zvols, provide block devices backed by the ZFS storage pool.

These can be presented to other systems through iSCSI.

This allows a consuming system to see a remote block device while the underlying allocation, integrity and snapshot behaviour remain managed by ZFS on HG000009.

The separation is useful, but it creates responsibilities on both sides.

The storage server owns the zvol and its availability. The client may place its own filesystem or data structure on the exported block device.

That means a storage-level snapshot may capture the block state without understanding whether the client application had completed its writes or reached a consistent transaction boundary.

Crash consistency and application consistency are not identical.

Where application-consistent recovery is required, snapshot operations may need coordination with the client or application.

SMB shares

Where file-level access is appropriate, ZFS datasets can support SMB shares.

The consumer interacts with files and directories rather than a raw block device.

This model is suitable for shared documents, general file storage and workloads that benefit from centralised file-level access and permissions.

SMB and iSCSI serve different purposes:

• SMB provides a shared filesystem service;

• iSCSI provides a remote block device.

Choosing between them should depend on the consumer’s requirements—not on which service is easier to enable.

Snapshots as recovery points

ZFS snapshots provide a lightweight way to preserve the state of a dataset or volume at a particular moment.

Because ZFS uses copy-on-write, a snapshot initially requires little additional storage. Space consumption grows as the live data diverges from the captured state.

Snapshots can support:

• recovery of accidentally deleted files;

• rollback after an unsuccessful change;

• comparison between states;

• consistent replication points;

• and short-term operational recovery.

They are extremely useful, but they require governance.

Without retention rules, snapshots can consume increasing amounts of capacity. A large number of snapshots can also make operational management more difficult.

Each snapshot policy should define:

• why snapshots are being created;

• how frequently they are created;

• how long they are retained;

• which datasets are included;

• and how recovery is tested.

Snapshots are not backups

A snapshot remains dependent on the storage pool that contains it.

If the pool is destroyed, the snapshot may be destroyed with it.

This means snapshots should not automatically be described as backups.

A robust protection strategy normally distinguishes between:

• local snapshots for rapid recovery;

• replication to another system;

• independent backups;

• and, where required, off-site or offline copies.

Each protects against a different failure.

Local snapshots may protect against accidental deletion. Replication may protect against failure of the primary node. An independent backup may protect against administrative error, corruption propagated through replication or the loss of the complete storage environment.

The required combination depends on the value of the data and the acceptable recovery objectives.

Replication

ZFS supports efficient replication through snapshots and incremental send-and-receive operations.

After an initial transfer, subsequent replication can send the changes between snapshots instead of retransmitting the complete dataset.

This creates a useful mechanism for maintaining another copy of storage state.

Replication still requires design decisions:

• What is replicated?

• How frequently?

• To which destination?

• How is the transfer protected?

• How is failure detected?

• How much history is retained?

• Can the replicated data be restored independently?

• What happens if undesirable changes are replicated?

A replication process that has never been tested in recovery is only evidence that data moved—not that the service can be restored.

Scrubs and continuous verification

Checksums are most useful when stored data is read and verified.

A ZFS scrub systematically reads the pool’s data and validates it against the stored checksums.

When redundancy is available, ZFS can repair corrupted copies using valid data from another location.

Regular scrubbing helps detect problems before the affected block is required during an incident or recovery.

The appropriate frequency depends on factors such as:

• storage size;

• device characteristics;

• workload;

• performance impact;

• and operational risk.

A scrub is not a substitute for monitoring physical-device health or maintaining backups. It is one part of the integrity model.

Explicit degraded states

One of my principal design goals is predictable failure behaviour.

If a device or path fails, the storage system should expose that state clearly.

Operators need to know:

• which pool is affected;

• which component failed;

• whether redundancy remains;

• whether data errors were detected;

• whether the system can continue safely;

• and which recovery action is required.

A system that continues operating in a degraded state can be useful because it preserves service while repairs are performed.

It can also create risk if the degraded condition is ignored.

The remaining redundancy may be reduced or absent. A second failure could result in data loss.

Degraded operation must therefore be visible and connected to an operational response.

Capacity is a reliability concern

Storage capacity is often treated as a financial or planning concern.

It is also an operational and reliability concern.

Copy-on-write filesystems need sufficient free space to operate effectively. Snapshots retain old blocks, and unexpected workload growth can consume capacity rapidly.

Monitoring only nominal free space is insufficient.

The operational model should consider:

• pool utilisation;

• dataset quotas;

• reservations;

• snapshot growth;

• replication backlog;

• workload trends;

• and the consequences of approaching exhaustion.

Capacity limits should trigger action before the storage system reaches a critical state.

Performance is not the only objective

This architecture was not designed primarily to maximise benchmark results.

Raw throughput and latency matter, but they are not the only qualities of useful storage.

The design also prioritises:

• understandable behaviour;

• data integrity;

• recoverability;

• visibility;

• controlled failure;

• and operational safety.

A benchmark can show how quickly a system performs a particular workload under particular conditions.

It does not show whether an engineer can recover a deleted dataset, explain a degraded pool or restore service after the loss of a component.

A storage system should be evaluated across its full lifecycle:

• normal operation;

• growth;

• maintenance;

• partial failure;

• repair;

• recovery;

• and eventual replacement.

Configuration changes require discipline

ZFS provides powerful administrative capabilities.

That power makes operational discipline important.

Before changing pool structure, replacing devices, modifying dataset properties or performing rollback operations, the engineer should understand:

• the current state;

• the intended outcome;

• the affected dependencies;

• the recovery path;

• and whether the action can be reversed.

Some storage decisions are difficult or impossible to undo without migrating data.

The correct time to consider those constraints is during design—not after capacity or failure pressure forces an immediate response.

Observability around ZFS

Treating ZFS as the storage control plane means observing it as a service.

Relevant information includes:

• pool health;

• device errors;

• capacity;

• fragmentation;

• I/O throughput and latency;

• cache behaviour;

• scrub status;

• snapshot growth;

• replication state;

• and back-end connectivity.

These signals should be connected to the services consuming the storage.

If a client reports increased latency, the investigation should be able to follow the path through the exported service, ZFS, the back-end connection and the physical device.

The control plane becomes useful when it makes that chain understandable.

Operational semantics must remain inspectable

One reason for selecting ZFS is that its storage model can be inspected.

Pools, virtual devices, datasets, volumes, snapshots and properties are represented explicitly.

This supports operational questions such as:

• Which devices form this pool?

• What redundancy exists?

• Which datasets are consuming capacity?

• Which snapshots retain space?

• When was the last scrub completed?

• Are errors accumulating?

• Which properties differ from the expected baseline?

The answers should not depend entirely on undocumented knowledge held by one person.

Storage that remains understandable under stress

The most important moment for a storage architecture is not when everything is healthy.

It is when:

• a device fails;

• a network path becomes unreliable;

• performance degrades;

• capacity approaches exhaustion;

• an operator makes a mistake;

• or data must be recovered quickly.

Under those conditions, complexity and ambiguity become operational risk.

By treating ZFS as the central control plane, storage behaviour becomes more explicit.

Devices are aggregated through a known model. Data integrity is verified. Snapshots and replication provide defined recovery mechanisms. Health and failure states are inspectable.

This does not remove the need for careful design, monitoring or backups.

It creates a coherent place in which those responsibilities can be managed.

The objective is not storage that merely performs well when healthy.

It is storage that remains understandable, recoverable and operationally safe when conditions are no longer ideal.

AI Assistance Disclosure

This article is based on my original ideas, experience, analysis and conclusions. Artificial intelligence tools were subsequently used as editorial and research assistants to review grammar and wording, improve structure and presentation, organise some arguments into clearer logical sections, and help review references to legal, regulatory and technical concepts.

Where relevant, factual and regulatory references were checked against the sources cited in the article. AI assistance does not replace professional legal, regulatory, financial or technical advice, and the final selection, interpretation, opinions and conclusions presented here remain my own.