Minecraft NBT (Named Binary Tag): Java parsing library

A structured binary format native to Minecraft for saving game data and transferring it over the network (in multiplayer), such as player data (<player>.dat; contains e.g. player's inventory and location), saved worlds (level.dat and Chunk format), list of saved multiplayer servers (servers.dat) and so on - see https://minecraft.gamepedia.com/NBT_format#Uses.

The entire file should be gzip-compressed (in accordance with the original specification NBT.txt by Notch), but can also be compressed with zlib or uncompressed.

This spec can only handle uncompressed NBT data, so be sure to first detect what type of data you are dealing with. You can use the Unix file command to do this (file-5.20 or later is required; older versions do not recognize zlib-compressed data and return application/octet-stream instead):

file --brief --mime-type input-unknown.nbt

If it says:

  • application/x-gzip or application/gzip (since file-5.37), you can decompress it by
    • gunzip -c input-gzip.nbt > output.nbt or
    • python3 -c "import sys, gzip; sys.stdout.buffer.write( gzip.decompress(sys.stdin.buffer.read()) )" < input-gzip.nbt > output.nbt
  • application/zlib, you can use
    • openssl zlib -d -in input-zlib.nbt -out output.nbt (does not work on most systems)
    • python3 -c "import sys, zlib; sys.stdout.buffer.write( zlib.decompress(sys.stdin.buffer.read()) )" < input-zlib.nbt > output.nbt
  • something else (especially image/x-pcx and application/octet-stream), it is most likely already uncompressed.

The file output.nbt generated by one of the above commands can already be processed with this Kaitai Struct specification.

This spec only implements the Java edition format. There is also a Bedrock edition NBT format, which uses little-endian encoding and has a few other differences, but it isn't as popular as the Java edition format.

Implementation note: strings in TAG_String are incorrectly decoded with standard UTF-8, while they are encoded in Modified UTF-8 (MUTF-8). That's because MUTF-8 is not supported natively by most target languages, and thus one must use external libraries to achieve a fully-compliant decoder. But decoding in standard UTF-8 is still better than nothing, and it usually works fine.

All Unicode code points with incompatible representations in MUTF-8 and UTF-8 are U+0000 (NUL), U+D800-U+DFFF (High and Low Surrogates) and U+10000-U+10FFFF (all Supplementary Planes; includes e.g. emoticons, pictograms). A MUTF-8-encoded string containing these code points cannot be successfully decoded as UTF-8. The behavior in this case depends on the target language - usually an exception is thrown, or the bytes that are not valid UTF-8 are replaced or ignored.

Sample files:

Application

Minecraft

File extension

["nbt", "dat", "schematic", "schem"]

KS implementation details

License: CC0-1.0

References

This page hosts a formal specification of Minecraft NBT (Named Binary Tag) using Kaitai Struct. This specification can be automatically translated into a variety of programming languages to get a parsing library.

Usage

Runtime library

All parsing code for Java generated by Kaitai Struct depends on the Java runtime library. You have to install it before you can parse data.

The Java runtime library is published in the Maven Central Repository. Refer to the artifact page for instructions how to add it into your project with the build tool that you use.

Code

Parse a local file and get structure in memory:

MinecraftNbt data = MinecraftNbt.fromFile("path/to/local/file.["nbt", "dat", "schematic", "schem"]");

Or parse structure from a byte array:

byte[] someArray = new byte[] { ... };
MinecraftNbt data = new MinecraftNbt(new ByteBufferKaitaiStream(someArray));

After that, one can get various attributes from the structure by invoking getter methods like:

data.rootCheck() // => get root check

Java source code to parse Minecraft NBT (Named Binary Tag)

MinecraftNbt.java

// This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild

import io.kaitai.struct.ByteBufferKaitaiStream;
import io.kaitai.struct.KaitaiStruct;
import io.kaitai.struct.KaitaiStream;
import java.io.IOException;
import java.util.Map;
import java.util.HashMap;
import java.util.ArrayList;
import java.nio.charset.Charset;


/**
 * A structured binary format native to Minecraft for saving game data and transferring
 * it over the network (in multiplayer), such as player data
 * ([`<player>.dat`](https://minecraft.gamepedia.com/Player.dat_format); contains
 * e.g. player's inventory and location), saved worlds
 * ([`level.dat`](
 *   https://minecraft.gamepedia.com/Java_Edition_level_format#level.dat_format
 * ) and [Chunk format](https://minecraft.gamepedia.com/Chunk_format#NBT_structure)),
 * list of saved multiplayer servers
 * ([`servers.dat`](https://minecraft.gamepedia.com/Servers.dat_format)) and so on -
 * see <https://minecraft.gamepedia.com/NBT_format#Uses>.
 * 
 * The entire file should be _gzip_-compressed (in accordance with the original
 * specification [NBT.txt](
 *   https://web.archive.org/web/20110723210920/https://www.minecraft.net/docs/NBT.txt
 * ) by Notch), but can also be compressed with _zlib_ or uncompressed.
 * 
 * This spec can only handle uncompressed NBT data, so be sure to first detect
 * what type of data you are dealing with. You can use the Unix `file` command
 * to do this (`file-5.20` or later is required; older versions do not recognize
 * _zlib_-compressed data and return `application/octet-stream` instead):
 * 
 * ```shell
 * file --brief --mime-type input-unknown.nbt
 * ```
 * 
 * If it says:
 * 
 *   * `application/x-gzip` or `application/gzip` (since `file-5.37`), you can decompress it by
 *     * `gunzip -c input-gzip.nbt > output.nbt` or
 *     * `python3 -c "import sys, gzip; sys.stdout.buffer.write(
 *       gzip.decompress(sys.stdin.buffer.read()) )" < input-gzip.nbt > output.nbt`
 *   * `application/zlib`, you can use
 *     * `openssl zlib -d -in input-zlib.nbt -out output.nbt` (does not work on most systems)
 *     * `python3 -c "import sys, zlib; sys.stdout.buffer.write(
 *       zlib.decompress(sys.stdin.buffer.read()) )" < input-zlib.nbt > output.nbt`
 *   * something else (especially `image/x-pcx` and `application/octet-stream`),
 *     it is most likely already uncompressed.
 * 
 * The file `output.nbt` generated by one of the above commands can already be
 * processed with this Kaitai Struct specification.
 * 
 * This spec **only** implements the Java edition format. There is also
 * a [Bedrock edition](https://wiki.vg/NBT#Bedrock_edition) NBT format,
 * which uses little-endian encoding and has a few other differences, but it isn't
 * as popular as the Java edition format.
 * 
 * **Implementation note:** strings in `TAG_String` are incorrectly decoded with
 * standard UTF-8, while they are encoded in [**Modified UTF-8**](
 *   https://docs.oracle.com/javase/8/docs/api/java/io/DataInput.html#modified-utf-8
 * ) (MUTF-8). That's because MUTF-8 is not supported natively by most target
 * languages, and thus one must use external libraries to achieve a fully-compliant
 * decoder. But decoding in standard UTF-8 is still better than nothing, and
 * it usually works fine.
 * 
 * All Unicode code points with incompatible representations in MUTF-8 and UTF-8 are
 * U+0000 (_NUL_), U+D800-U+DFFF (_High_ and _Low Surrogates_) and U+10000-U+10FFFF
 * (all _Supplementary_ Planes; includes e.g. emoticons, pictograms).
 * A _MUTF-8_-encoded string containing these code points cannot be successfully
 * decoded as UTF-8. The behavior in this case depends on the target language -
 * usually an exception is thrown, or the bytes that are not valid UTF-8
 * are replaced or ignored.
 * 
 * **Sample files:**
 * 
 *   * <https://wiki.vg/NBT#Download>
 *   * <https://github.com/twoolie/NBT/blob/f9e892e/tests/world_test/data/scoreboard.dat>
 *   * <https://github.com/chmod222/cNBT/tree/3f74b69/testdata>
 *   * <https://github.com/PistonDevelopers/hematite_nbt/tree/0b85f89/tests>
 * @see <a href="https://wiki.vg/NBT">Source</a>
 * @see <a href="https://web.archive.org/web/20110723210920/https://www.minecraft.net/docs/NBT.txt">Source</a>
 * @see <a href="https://minecraft.gamepedia.com/NBT_format">Source</a>
 */
public class MinecraftNbt extends KaitaiStruct {
    public static MinecraftNbt fromFile(String fileName) throws IOException {
        return new MinecraftNbt(new ByteBufferKaitaiStream(fileName));
    }

    public enum Tag {
        END(0),
        BYTE(1),
        SHORT(2),
        INT(3),
        LONG(4),
        FLOAT(5),
        DOUBLE(6),
        BYTE_ARRAY(7),
        STRING(8),
        LIST(9),
        COMPOUND(10),
        INT_ARRAY(11),
        LONG_ARRAY(12);

        private final long id;
        Tag(long id) { this.id = id; }
        public long id() { return id; }
        private static final Map<Long, Tag> byId = new HashMap<Long, Tag>(13);
        static {
            for (Tag e : Tag.values())
                byId.put(e.id(), e);
        }
        public static Tag byId(long id) { return byId.get(id); }
    }

    public MinecraftNbt(KaitaiStream _io) {
        this(_io, null, null);
    }

    public MinecraftNbt(KaitaiStream _io, KaitaiStruct _parent) {
        this(_io, _parent, null);
    }

    public MinecraftNbt(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
        super(_io);
        this._parent = _parent;
        this._root = _root == null ? this : _root;
        _read();
    }
    private void _read() {
        if ( ((rootType() == Tag.END) && (false)) ) {
            this.rootCheck = this._io.readBytes(0);
        }
        this.root = new NamedTag(this._io, this, _root);
    }
    public static class TagLongArray extends KaitaiStruct {
        public static TagLongArray fromFile(String fileName) throws IOException {
            return new TagLongArray(new ByteBufferKaitaiStream(fileName));
        }

        public TagLongArray(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagLongArray(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagLongArray(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.numTags = this._io.readS4be();
            tags = new ArrayList<Long>(((Number) (numTags())).intValue());
            for (int i = 0; i < numTags(); i++) {
                this.tags.add(this._io.readS8be());
            }
        }
        private Tag tagsType;
        public Tag tagsType() {
            if (this.tagsType != null)
                return this.tagsType;
            this.tagsType = MinecraftNbt.Tag.LONG;
            return this.tagsType;
        }
        private int numTags;
        private ArrayList<Long> tags;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public int numTags() { return numTags; }
        public ArrayList<Long> tags() { return tags; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class TagByteArray extends KaitaiStruct {
        public static TagByteArray fromFile(String fileName) throws IOException {
            return new TagByteArray(new ByteBufferKaitaiStream(fileName));
        }

        public TagByteArray(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagByteArray(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagByteArray(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.lenData = this._io.readS4be();
            this.data = this._io.readBytes(lenData());
        }
        private int lenData;
        private byte[] data;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public int lenData() { return lenData; }
        public byte[] data() { return data; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class TagIntArray extends KaitaiStruct {
        public static TagIntArray fromFile(String fileName) throws IOException {
            return new TagIntArray(new ByteBufferKaitaiStream(fileName));
        }

        public TagIntArray(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagIntArray(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagIntArray(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.numTags = this._io.readS4be();
            tags = new ArrayList<Integer>(((Number) (numTags())).intValue());
            for (int i = 0; i < numTags(); i++) {
                this.tags.add(this._io.readS4be());
            }
        }
        private Tag tagsType;
        public Tag tagsType() {
            if (this.tagsType != null)
                return this.tagsType;
            this.tagsType = MinecraftNbt.Tag.INT;
            return this.tagsType;
        }
        private int numTags;
        private ArrayList<Integer> tags;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public int numTags() { return numTags; }
        public ArrayList<Integer> tags() { return tags; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class TagList extends KaitaiStruct {
        public static TagList fromFile(String fileName) throws IOException {
            return new TagList(new ByteBufferKaitaiStream(fileName));
        }

        public TagList(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagList(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagList(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.tagsType = MinecraftNbt.Tag.byId(this._io.readU1());
            this.numTags = this._io.readS4be();
            tags = new ArrayList<Object>(((Number) (numTags())).intValue());
            for (int i = 0; i < numTags(); i++) {
                {
                    Tag on = tagsType();
                    if (on != null) {
                        switch (tagsType()) {
                        case LONG_ARRAY: {
                            this.tags.add(new TagLongArray(this._io, this, _root));
                            break;
                        }
                        case COMPOUND: {
                            this.tags.add(new TagCompound(this._io, this, _root));
                            break;
                        }
                        case DOUBLE: {
                            this.tags.add((Object) (this._io.readF8be()));
                            break;
                        }
                        case LIST: {
                            this.tags.add(new TagList(this._io, this, _root));
                            break;
                        }
                        case FLOAT: {
                            this.tags.add((Object) (this._io.readF4be()));
                            break;
                        }
                        case SHORT: {
                            this.tags.add((Object) (this._io.readS2be()));
                            break;
                        }
                        case INT: {
                            this.tags.add((Object) (this._io.readS4be()));
                            break;
                        }
                        case BYTE_ARRAY: {
                            this.tags.add(new TagByteArray(this._io, this, _root));
                            break;
                        }
                        case BYTE: {
                            this.tags.add((Object) (this._io.readS1()));
                            break;
                        }
                        case INT_ARRAY: {
                            this.tags.add(new TagIntArray(this._io, this, _root));
                            break;
                        }
                        case STRING: {
                            this.tags.add(new TagString(this._io, this, _root));
                            break;
                        }
                        case LONG: {
                            this.tags.add((Object) (this._io.readS8be()));
                            break;
                        }
                        }
                    }
                }
            }
        }
        private Tag tagsType;
        private int numTags;
        private ArrayList<Object> tags;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public Tag tagsType() { return tagsType; }
        public int numTags() { return numTags; }
        public ArrayList<Object> tags() { return tags; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class TagString extends KaitaiStruct {
        public static TagString fromFile(String fileName) throws IOException {
            return new TagString(new ByteBufferKaitaiStream(fileName));
        }

        public TagString(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagString(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagString(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.lenData = this._io.readU2be();
            this.data = new String(this._io.readBytes(lenData()), Charset.forName("utf-8"));
        }
        private int lenData;
        private String data;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;

        /**
         * unsigned according to https://wiki.vg/NBT#Specification
         */
        public int lenData() { return lenData; }
        public String data() { return data; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class TagCompound extends KaitaiStruct {
        public static TagCompound fromFile(String fileName) throws IOException {
            return new TagCompound(new ByteBufferKaitaiStream(fileName));
        }

        public TagCompound(KaitaiStream _io) {
            this(_io, null, null);
        }

        public TagCompound(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public TagCompound(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.tags = new ArrayList<NamedTag>();
            {
                NamedTag _it;
                int i = 0;
                do {
                    _it = new NamedTag(this._io, this, _root);
                    this.tags.add(_it);
                    i++;
                } while (!(_it.isTagEnd()));
            }
        }
        private Integer dumpNumTags;
        public Integer dumpNumTags() {
            if (this.dumpNumTags != null)
                return this.dumpNumTags;
            int _tmp = (int) ((tags().size() - ( ((tags().size() >= 1) && (tags().get(tags().size() - 1).isTagEnd()))  ? 1 : 0)));
            this.dumpNumTags = _tmp;
            return this.dumpNumTags;
        }
        private ArrayList<NamedTag> tags;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public ArrayList<NamedTag> tags() { return tags; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    public static class NamedTag extends KaitaiStruct {
        public static NamedTag fromFile(String fileName) throws IOException {
            return new NamedTag(new ByteBufferKaitaiStream(fileName));
        }

        public NamedTag(KaitaiStream _io) {
            this(_io, null, null);
        }

        public NamedTag(KaitaiStream _io, KaitaiStruct _parent) {
            this(_io, _parent, null);
        }

        public NamedTag(KaitaiStream _io, KaitaiStruct _parent, MinecraftNbt _root) {
            super(_io);
            this._parent = _parent;
            this._root = _root;
            _read();
        }
        private void _read() {
            this.type = MinecraftNbt.Tag.byId(this._io.readU1());
            if (!(isTagEnd())) {
                this.name = new TagString(this._io, this, _root);
            }
            if (!(isTagEnd())) {
                {
                    Tag on = type();
                    if (on != null) {
                        switch (type()) {
                        case LONG_ARRAY: {
                            this.payload = new TagLongArray(this._io, this, _root);
                            break;
                        }
                        case COMPOUND: {
                            this.payload = new TagCompound(this._io, this, _root);
                            break;
                        }
                        case DOUBLE: {
                            this.payload = (Object) (this._io.readF8be());
                            break;
                        }
                        case LIST: {
                            this.payload = new TagList(this._io, this, _root);
                            break;
                        }
                        case FLOAT: {
                            this.payload = (Object) (this._io.readF4be());
                            break;
                        }
                        case SHORT: {
                            this.payload = (Object) (this._io.readS2be());
                            break;
                        }
                        case INT: {
                            this.payload = (Object) (this._io.readS4be());
                            break;
                        }
                        case BYTE_ARRAY: {
                            this.payload = new TagByteArray(this._io, this, _root);
                            break;
                        }
                        case BYTE: {
                            this.payload = (Object) (this._io.readS1());
                            break;
                        }
                        case INT_ARRAY: {
                            this.payload = new TagIntArray(this._io, this, _root);
                            break;
                        }
                        case STRING: {
                            this.payload = new TagString(this._io, this, _root);
                            break;
                        }
                        case LONG: {
                            this.payload = (Object) (this._io.readS8be());
                            break;
                        }
                        }
                    }
                }
            }
        }
        private Boolean isTagEnd;
        public Boolean isTagEnd() {
            if (this.isTagEnd != null)
                return this.isTagEnd;
            boolean _tmp = (boolean) (type() == MinecraftNbt.Tag.END);
            this.isTagEnd = _tmp;
            return this.isTagEnd;
        }
        private Tag type;
        private TagString name;
        private Object payload;
        private MinecraftNbt _root;
        private KaitaiStruct _parent;
        public Tag type() { return type; }
        public TagString name() { return name; }
        public Object payload() { return payload; }
        public MinecraftNbt _root() { return _root; }
        public KaitaiStruct _parent() { return _parent; }
    }
    private Tag rootType;
    public Tag rootType() {
        if (this.rootType != null)
            return this.rootType;
        long _pos = this._io.pos();
        this._io.seek(0);
        this.rootType = Tag.byId(this._io.readU1());
        this._io.seek(_pos);
        if (!(rootType() == Tag.COMPOUND)) {
            throw new KaitaiStream.ValidationNotEqualError(Tag.COMPOUND, rootType(), _io(), "/instances/root_type");
        }
        return this.rootType;
    }
    private byte[] rootCheck;
    private NamedTag root;
    private MinecraftNbt _root;
    private KaitaiStruct _parent;
    public byte[] rootCheck() { return rootCheck; }
    public NamedTag root() { return root; }
    public MinecraftNbt _root() { return _root; }
    public KaitaiStruct _parent() { return _parent; }
}