Minecraft NBT (Named Binary Tag): Perl parsing library

A structured binary format native to Minecraft for saving game data and transferring it over the network (in multiplayer), such as player data (<player>.dat; contains e.g. player's inventory and location), saved worlds (level.dat and Chunk format), list of saved multiplayer servers (servers.dat) and so on - see https://minecraft.wiki/w/NBT_format#Uses.

The entire file should be gzip-compressed (in accordance with the original specification NBT.txt by Notch), but can also be compressed with zlib or uncompressed.

This spec can only handle uncompressed NBT data, so be sure to first detect what type of data you are dealing with. You can use the Unix file command to do this (file-5.20 or later is required; older versions do not recognize zlib-compressed data and return application/octet-stream instead):

file --brief --mime-type input-unknown.nbt

If it says:

application/x-gzip or application/gzip (since file-5.37), you can decompress it by
- gunzip -c input-gzip.nbt > output.nbt or
- python3 -c "import sys, gzip; sys.stdout.buffer.write( gzip.decompress(sys.stdin.buffer.read()) )" < input-gzip.nbt > output.nbt
application/zlib, you can use
- openssl zlib -d -in input-zlib.nbt -out output.nbt (does not work on most systems)
- python3 -c "import sys, zlib; sys.stdout.buffer.write( zlib.decompress(sys.stdin.buffer.read()) )" < input-zlib.nbt > output.nbt
something else (especially image/x-pcx and application/octet-stream), it is most likely already uncompressed.

The file output.nbt generated by one of the above commands can already be processed with this Kaitai Struct specification.

This spec only implements the Java edition format. There is also a Bedrock edition NBT format, which uses little-endian encoding and has a few other differences, but it isn't as popular as the Java edition format.

Implementation note: strings in TAG_String are incorrectly decoded with standard UTF-8, while they are encoded in Modified UTF-8 (MUTF-8). That's because MUTF-8 is not supported natively by most target languages, and thus one must use external libraries to achieve a fully-compliant decoder. But decoding in standard UTF-8 is still better than nothing, and it usually works fine.

All Unicode code points with incompatible representations in MUTF-8 and UTF-8 are U+0000 (NUL), U+D800-U+DFFF (High and Low Surrogates) and U+10000-U+10FFFF (all Supplementary Planes; includes e.g. emoticons, pictograms). A MUTF-8-encoded string containing these code points cannot be successfully decoded as UTF-8. The behavior in this case depends on the target language - usually an exception is thrown, or the bytes that are not valid UTF-8 are replaced or ignored.

Sample files:

Application

Minecraft

File extension

["nbt", "dat", "schematic", "schem"]

KS implementation details

License: CC0-1.0

References

Minecraft NBT format in Just Solve the File Format Problem

This page hosts a formal specification of Minecraft NBT (Named Binary Tag) using Kaitai Struct. This specification can be automatically translated into a variety of programming languages to get a parsing library.

Perl source code to parse Minecraft NBT (Named Binary Tag)

MinecraftNbt.pm

Download

# This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild

use strict;
use warnings;
use IO::KaitaiStruct 0.009_000;
use Encode;

########################################################################
package MinecraftNbt;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

our $TAG_END = 0;
our $TAG_BYTE = 1;
our $TAG_SHORT = 2;
our $TAG_INT = 3;
our $TAG_LONG = 4;
our $TAG_FLOAT = 5;
our $TAG_DOUBLE = 6;
our $TAG_BYTE_ARRAY = 7;
our $TAG_STRING = 8;
our $TAG_LIST = 9;
our $TAG_COMPOUND = 10;
our $TAG_INT_ARRAY = 11;
our $TAG_LONG_ARRAY = 12;

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    if ( (($self->root_type() == $MinecraftNbt::TAG_END) && (0)) ) {
        $self->{root_check} = $self->{_io}->read_bytes(0);
    }
    $self->{root} = MinecraftNbt::NamedTag->new($self->{_io}, $self, $self->{_root});
}

sub root_type {
    my ($self) = @_;
    return $self->{root_type} if ($self->{root_type});
    my $_pos = $self->{_io}->pos();
    $self->{_io}->seek(0);
    $self->{root_type} = $self->{_io}->read_u1();
    $self->{_io}->seek($_pos);
    return $self->{root_type};
}

sub root_check {
    my ($self) = @_;
    return $self->{root_check};
}

sub root {
    my ($self) = @_;
    return $self->{root};
}

########################################################################
package MinecraftNbt::TagLongArray;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{num_tags} = $self->{_io}->read_s4be();
    $self->{tags} = ();
    my $n_tags = $self->num_tags();
    for (my $i = 0; $i < $n_tags; $i++) {
        push @{$self->{tags}}, $self->{_io}->read_s8be();
    }
}

sub tags_type {
    my ($self) = @_;
    return $self->{tags_type} if ($self->{tags_type});
    $self->{tags_type} = $MinecraftNbt::TAG_LONG;
    return $self->{tags_type};
}

sub num_tags {
    my ($self) = @_;
    return $self->{num_tags};
}

sub tags {
    my ($self) = @_;
    return $self->{tags};
}

########################################################################
package MinecraftNbt::TagByteArray;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{len_data} = $self->{_io}->read_s4be();
    $self->{data} = $self->{_io}->read_bytes($self->len_data());
}

sub len_data {
    my ($self) = @_;
    return $self->{len_data};
}

sub data {
    my ($self) = @_;
    return $self->{data};
}

########################################################################
package MinecraftNbt::TagIntArray;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{num_tags} = $self->{_io}->read_s4be();
    $self->{tags} = ();
    my $n_tags = $self->num_tags();
    for (my $i = 0; $i < $n_tags; $i++) {
        push @{$self->{tags}}, $self->{_io}->read_s4be();
    }
}

sub tags_type {
    my ($self) = @_;
    return $self->{tags_type} if ($self->{tags_type});
    $self->{tags_type} = $MinecraftNbt::TAG_INT;
    return $self->{tags_type};
}

sub num_tags {
    my ($self) = @_;
    return $self->{num_tags};
}

sub tags {
    my ($self) = @_;
    return $self->{tags};
}

########################################################################
package MinecraftNbt::TagList;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{tags_type} = $self->{_io}->read_u1();
    $self->{num_tags} = $self->{_io}->read_s4be();
    $self->{tags} = ();
    my $n_tags = $self->num_tags();
    for (my $i = 0; $i < $n_tags; $i++) {
        my $_on = $self->tags_type();
        if ($_on == $MinecraftNbt::TAG_LONG_ARRAY) {
            push @{$self->{tags}}, MinecraftNbt::TagLongArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_COMPOUND) {
            push @{$self->{tags}}, MinecraftNbt::TagCompound->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_DOUBLE) {
            push @{$self->{tags}}, $self->{_io}->read_f8be();
        }
        elsif ($_on == $MinecraftNbt::TAG_LIST) {
            push @{$self->{tags}}, MinecraftNbt::TagList->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_FLOAT) {
            push @{$self->{tags}}, $self->{_io}->read_f4be();
        }
        elsif ($_on == $MinecraftNbt::TAG_SHORT) {
            push @{$self->{tags}}, $self->{_io}->read_s2be();
        }
        elsif ($_on == $MinecraftNbt::TAG_INT) {
            push @{$self->{tags}}, $self->{_io}->read_s4be();
        }
        elsif ($_on == $MinecraftNbt::TAG_BYTE_ARRAY) {
            push @{$self->{tags}}, MinecraftNbt::TagByteArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_BYTE) {
            push @{$self->{tags}}, $self->{_io}->read_s1();
        }
        elsif ($_on == $MinecraftNbt::TAG_INT_ARRAY) {
            push @{$self->{tags}}, MinecraftNbt::TagIntArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_STRING) {
            push @{$self->{tags}}, MinecraftNbt::TagString->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_LONG) {
            push @{$self->{tags}}, $self->{_io}->read_s8be();
        }
    }
}

sub tags_type {
    my ($self) = @_;
    return $self->{tags_type};
}

sub num_tags {
    my ($self) = @_;
    return $self->{num_tags};
}

sub tags {
    my ($self) = @_;
    return $self->{tags};
}

########################################################################
package MinecraftNbt::TagString;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{len_data} = $self->{_io}->read_u2be();
    $self->{data} = Encode::decode("utf-8", $self->{_io}->read_bytes($self->len_data()));
}

sub len_data {
    my ($self) = @_;
    return $self->{len_data};
}

sub data {
    my ($self) = @_;
    return $self->{data};
}

########################################################################
package MinecraftNbt::TagCompound;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{tags} = ();
    do {
        $_ = MinecraftNbt::NamedTag->new($self->{_io}, $self, $self->{_root});
        push @{$self->{tags}}, $_;
    } until ($_->is_tag_end());
}

sub dump_num_tags {
    my ($self) = @_;
    return $self->{dump_num_tags} if ($self->{dump_num_tags});
    $self->{dump_num_tags} = (scalar(@{$self->tags()}) - ( ((scalar(@{$self->tags()}) >= 1) && (@{$self->tags()}[-1]->is_tag_end()))  ? 1 : 0));
    return $self->{dump_num_tags};
}

sub tags {
    my ($self) = @_;
    return $self->{tags};
}

########################################################################
package MinecraftNbt::NamedTag;

our @ISA = 'IO::KaitaiStruct::Struct';

sub from_file {
    my ($class, $filename) = @_;
    my $fd;

    open($fd, '<', $filename) or return undef;
    binmode($fd);
    return new($class, IO::KaitaiStruct::Stream->new($fd));
}

sub new {
    my ($class, $_io, $_parent, $_root) = @_;
    my $self = IO::KaitaiStruct::Struct->new($_io);

    bless $self, $class;
    $self->{_parent} = $_parent;
    $self->{_root} = $_root || $self;;

    $self->_read();

    return $self;
}

sub _read {
    my ($self) = @_;

    $self->{type} = $self->{_io}->read_u1();
    if (!($self->is_tag_end())) {
        $self->{name} = MinecraftNbt::TagString->new($self->{_io}, $self, $self->{_root});
    }
    if (!($self->is_tag_end())) {
        my $_on = $self->type();
        if ($_on == $MinecraftNbt::TAG_LONG_ARRAY) {
            $self->{payload} = MinecraftNbt::TagLongArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_COMPOUND) {
            $self->{payload} = MinecraftNbt::TagCompound->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_DOUBLE) {
            $self->{payload} = $self->{_io}->read_f8be();
        }
        elsif ($_on == $MinecraftNbt::TAG_LIST) {
            $self->{payload} = MinecraftNbt::TagList->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_FLOAT) {
            $self->{payload} = $self->{_io}->read_f4be();
        }
        elsif ($_on == $MinecraftNbt::TAG_SHORT) {
            $self->{payload} = $self->{_io}->read_s2be();
        }
        elsif ($_on == $MinecraftNbt::TAG_INT) {
            $self->{payload} = $self->{_io}->read_s4be();
        }
        elsif ($_on == $MinecraftNbt::TAG_BYTE_ARRAY) {
            $self->{payload} = MinecraftNbt::TagByteArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_BYTE) {
            $self->{payload} = $self->{_io}->read_s1();
        }
        elsif ($_on == $MinecraftNbt::TAG_INT_ARRAY) {
            $self->{payload} = MinecraftNbt::TagIntArray->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_STRING) {
            $self->{payload} = MinecraftNbt::TagString->new($self->{_io}, $self, $self->{_root});
        }
        elsif ($_on == $MinecraftNbt::TAG_LONG) {
            $self->{payload} = $self->{_io}->read_s8be();
        }
    }
}

sub is_tag_end {
    my ($self) = @_;
    return $self->{is_tag_end} if ($self->{is_tag_end});
    $self->{is_tag_end} = $self->type() == $MinecraftNbt::TAG_END;
    return $self->{is_tag_end};
}

sub type {
    my ($self) = @_;
    return $self->{type};
}

sub name {
    my ($self) = @_;
    return $self->{name};
}

sub payload {
    my ($self) = @_;
    return $self->{payload};
}

1;