TE (Terse Executable) file: C++11/STL parsing library

This type of executables could be found inside the UEFI firmware. The UEFI firmware is stored in SPI flash memory, which is a chip soldered on a system's motherboard. UEFI firmware is very modular: it usually contains dozens, if not hundreds, of executables. To store all these separates files, the firmware is laid out in volumes using the Firmware File System (FFS), a file system specifically designed to store firmware images. The volumes contain files that are identified by GUIDs and each of these files contain one or more sections holding the data. One of these sections contains the actual executable image. Most of the executable images follow the PE format. However, some of them follow the TE format.

The Terse Executable (TE) image format was created as a mechanism to reduce the overhead of the PE/COFF headers in PE32/PE32+ images, resulting in a corresponding reduction of image sizes for executables running in the PI (Platform Initialization) Architecture environment. Reducing image size provides an opportunity for use of a smaller system flash part.

So the TE format is basically a stripped version of PE.

Application

UEFI

File extension

["efi", "te"]

KS implementation details

License: CC0-1.0

Minimal Kaitai Struct required: 0.7

References

Wikidata Q83443959

This page hosts a formal specification of TE (Terse Executable) file using Kaitai Struct. This specification can be automatically translated into a variety of programming languages to get a parsing library.

Usage

Runtime library

All parsing code for C++11/STL generated by Kaitai Struct depends on the C++/STL runtime library. You have to install it before you can parse data.

For C++, the easiest way is to clone the runtime library sources and build them along with your project.

Code

Using Kaitai Struct in C++/STL usually consists of 3 steps.

We need to create an STL input stream (std::istream). One can open local file for that, or use existing std::string or char* buffer.

#include <fstream>

std::ifstream is("path/to/local/file.efi", std::ifstream::binary);

#include <sstream>

std::istringstream is(str);

#include <sstream>

const char buf[] = { ... };
std::string str(buf, sizeof buf);
std::istringstream is(str);

We need to wrap our input stream into Kaitai stream:

#include "kaitai/kaitaistream.h"

kaitai::kstream ks(&is);

And finally, we can invoke the parsing:
```
uefi_te_t data(&ks);
```

After that, one can get various attributes from the structure by invoking getter methods like:

data.te_hdr() // => get te hdr

C++11/STL source code to parse TE (Terse Executable) file

uefi_te.h

Download

#pragma once

// This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild

#include "kaitai/kaitaistruct.h"
#include <stdint.h>
#include <memory>
#include <vector>

#if KAITAI_STRUCT_VERSION < 9000L
#error "Incompatible Kaitai Struct C++/STL API: version 0.9 or later is required"
#endif

/**
 * This type of executables could be found inside the UEFI firmware. The UEFI
 * firmware is stored in SPI flash memory, which is a chip soldered on a
 * system's motherboard. UEFI firmware is very modular: it usually contains
 * dozens, if not hundreds, of executables. To store all these separates files,
 * the firmware is laid out in volumes using the Firmware File System (FFS), a
 * file system specifically designed to store firmware images. The volumes
 * contain files that are identified by GUIDs and each of these files contain
 * one or more sections holding the data. One of these sections contains the
 * actual executable image. Most of the executable images follow the PE format.
 * However, some of them follow the TE format.
 * 
 * The Terse Executable (TE) image format was created as a mechanism to reduce
 * the overhead of the PE/COFF headers in PE32/PE32+ images, resulting in a
 * corresponding reduction of image sizes for executables running in the PI
 * (Platform Initialization) Architecture environment. Reducing image size
 * provides an opportunity for use of a smaller system flash part.
 * 
 * So the TE format is basically a stripped version of PE.
 * \sa https://uefi.org/sites/default/files/resources/PI_Spec_1_6.pdf Source
 */

class uefi_te_t : public kaitai::kstruct {

public:
    class te_header_t;
    class header_data_dirs_t;
    class data_dir_t;
    class section_t;

    uefi_te_t(kaitai::kstream* p__io, kaitai::kstruct* p__parent = nullptr, uefi_te_t* p__root = nullptr);

private:
    void _read();
    void _clean_up();

public:
    ~uefi_te_t();

    class te_header_t : public kaitai::kstruct {

    public:

        enum machine_type_t {
            MACHINE_TYPE_UNKNOWN = 0,
            MACHINE_TYPE_I386 = 332,
            MACHINE_TYPE_R4000 = 358,
            MACHINE_TYPE_WCE_MIPS_V2 = 361,
            MACHINE_TYPE_ALPHA = 388,
            MACHINE_TYPE_SH3 = 418,
            MACHINE_TYPE_SH3_DSP = 419,
            MACHINE_TYPE_SH4 = 422,
            MACHINE_TYPE_SH5 = 424,
            MACHINE_TYPE_ARM = 448,
            MACHINE_TYPE_THUMB = 450,
            MACHINE_TYPE_ARM_NT = 452,
            MACHINE_TYPE_AM33 = 467,
            MACHINE_TYPE_POWERPC = 496,
            MACHINE_TYPE_POWERPC_FP = 497,
            MACHINE_TYPE_IA64 = 512,
            MACHINE_TYPE_MIPS16 = 614,
            MACHINE_TYPE_ALPHA64_OR_AXP64 = 644,
            MACHINE_TYPE_MIPS_FPU = 870,
            MACHINE_TYPE_MIPS16_FPU = 1126,
            MACHINE_TYPE_EBC = 3772,
            MACHINE_TYPE_RISCV32 = 20530,
            MACHINE_TYPE_RISCV64 = 20580,
            MACHINE_TYPE_RISCV128 = 20776,
            MACHINE_TYPE_LOONGARCH32 = 25138,
            MACHINE_TYPE_LOONGARCH64 = 25188,
            MACHINE_TYPE_AMD64 = 34404,
            MACHINE_TYPE_M32R = 36929,
            MACHINE_TYPE_ARM64 = 43620
        };

        enum subsystem_enum_t {
            SUBSYSTEM_ENUM_UNKNOWN = 0,
            SUBSYSTEM_ENUM_NATIVE = 1,
            SUBSYSTEM_ENUM_WINDOWS_GUI = 2,
            SUBSYSTEM_ENUM_WINDOWS_CUI = 3,
            SUBSYSTEM_ENUM_POSIX_CUI = 7,
            SUBSYSTEM_ENUM_WINDOWS_CE_GUI = 9,
            SUBSYSTEM_ENUM_EFI_APPLICATION = 10,
            SUBSYSTEM_ENUM_EFI_BOOT_SERVICE_DRIVER = 11,
            SUBSYSTEM_ENUM_EFI_RUNTIME_DRIVER = 12,
            SUBSYSTEM_ENUM_EFI_ROM = 13,
            SUBSYSTEM_ENUM_XBOX = 14,
            SUBSYSTEM_ENUM_WINDOWS_BOOT_APPLICATION = 16
        };

        te_header_t(kaitai::kstream* p__io, uefi_te_t* p__parent = nullptr, uefi_te_t* p__root = nullptr);

    private:
        void _read();
        void _clean_up();

    public:
        ~te_header_t();

    private:
        std::string m_magic;
        machine_type_t m_machine;
        uint8_t m_num_sections;
        subsystem_enum_t m_subsystem;
        uint16_t m_stripped_size;
        uint32_t m_entry_point_addr;
        uint32_t m_base_of_code;
        uint64_t m_image_base;
        std::unique_ptr<header_data_dirs_t> m_data_dirs;
        uefi_te_t* m__root;
        uefi_te_t* m__parent;

    public:
        std::string magic() const { return m_magic; }
        machine_type_t machine() const { return m_machine; }
        uint8_t num_sections() const { return m_num_sections; }
        subsystem_enum_t subsystem() const { return m_subsystem; }
        uint16_t stripped_size() const { return m_stripped_size; }
        uint32_t entry_point_addr() const { return m_entry_point_addr; }
        uint32_t base_of_code() const { return m_base_of_code; }
        uint64_t image_base() const { return m_image_base; }
        header_data_dirs_t* data_dirs() const { return m_data_dirs.get(); }
        uefi_te_t* _root() const { return m__root; }
        uefi_te_t* _parent() const { return m__parent; }
    };

    class header_data_dirs_t : public kaitai::kstruct {

    public:

        header_data_dirs_t(kaitai::kstream* p__io, uefi_te_t::te_header_t* p__parent = nullptr, uefi_te_t* p__root = nullptr);

    private:
        void _read();
        void _clean_up();

    public:
        ~header_data_dirs_t();

    private:
        std::unique_ptr<data_dir_t> m_base_relocation_table;
        std::unique_ptr<data_dir_t> m_debug;
        uefi_te_t* m__root;
        uefi_te_t::te_header_t* m__parent;

    public:
        data_dir_t* base_relocation_table() const { return m_base_relocation_table.get(); }
        data_dir_t* debug() const { return m_debug.get(); }
        uefi_te_t* _root() const { return m__root; }
        uefi_te_t::te_header_t* _parent() const { return m__parent; }
    };

    class data_dir_t : public kaitai::kstruct {

    public:

        data_dir_t(kaitai::kstream* p__io, uefi_te_t::header_data_dirs_t* p__parent = nullptr, uefi_te_t* p__root = nullptr);

    private:
        void _read();
        void _clean_up();

    public:
        ~data_dir_t();

    private:
        uint32_t m_virtual_address;
        uint32_t m_size;
        uefi_te_t* m__root;
        uefi_te_t::header_data_dirs_t* m__parent;

    public:
        uint32_t virtual_address() const { return m_virtual_address; }
        uint32_t size() const { return m_size; }
        uefi_te_t* _root() const { return m__root; }
        uefi_te_t::header_data_dirs_t* _parent() const { return m__parent; }
    };

    class section_t : public kaitai::kstruct {

    public:

        section_t(kaitai::kstream* p__io, uefi_te_t* p__parent = nullptr, uefi_te_t* p__root = nullptr);

    private:
        void _read();
        void _clean_up();

    public:
        ~section_t();

    private:
        bool f_body;
        std::string m_body;

    public:
        std::string body();

    private:
        std::string m_name;
        uint32_t m_virtual_size;
        uint32_t m_virtual_address;
        uint32_t m_size_of_raw_data;
        uint32_t m_pointer_to_raw_data;
        uint32_t m_pointer_to_relocations;
        uint32_t m_pointer_to_linenumbers;
        uint16_t m_num_relocations;
        uint16_t m_num_linenumbers;
        uint32_t m_characteristics;
        uefi_te_t* m__root;
        uefi_te_t* m__parent;

    public:
        std::string name() const { return m_name; }
        uint32_t virtual_size() const { return m_virtual_size; }
        uint32_t virtual_address() const { return m_virtual_address; }
        uint32_t size_of_raw_data() const { return m_size_of_raw_data; }
        uint32_t pointer_to_raw_data() const { return m_pointer_to_raw_data; }
        uint32_t pointer_to_relocations() const { return m_pointer_to_relocations; }
        uint32_t pointer_to_linenumbers() const { return m_pointer_to_linenumbers; }
        uint16_t num_relocations() const { return m_num_relocations; }
        uint16_t num_linenumbers() const { return m_num_linenumbers; }
        uint32_t characteristics() const { return m_characteristics; }
        uefi_te_t* _root() const { return m__root; }
        uefi_te_t* _parent() const { return m__parent; }
    };

private:
    std::unique_ptr<te_header_t> m_te_hdr;
    std::unique_ptr<std::vector<std::unique_ptr<section_t>>> m_sections;
    uefi_te_t* m__root;
    kaitai::kstruct* m__parent;
    std::string m__raw_te_hdr;
    std::unique_ptr<kaitai::kstream> m__io__raw_te_hdr;

public:
    te_header_t* te_hdr() const { return m_te_hdr.get(); }
    std::vector<std::unique_ptr<section_t>>* sections() const { return m_sections.get(); }
    uefi_te_t* _root() const { return m__root; }
    kaitai::kstruct* _parent() const { return m__parent; }
    std::string _raw_te_hdr() const { return m__raw_te_hdr; }
    kaitai::kstream* _io__raw_te_hdr() const { return m__io__raw_te_hdr.get(); }
};

uefi_te.cpp

Download

// This is a generated file! Please edit source .ksy file and use kaitai-struct-compiler to rebuild

#include "uefi_te.h"
#include "kaitai/exceptions.h"

uefi_te_t::uefi_te_t(kaitai::kstream* p__io, kaitai::kstruct* p__parent, uefi_te_t* p__root) : kaitai::kstruct(p__io) {
    m__parent = p__parent;
    m__root = this;
    m_te_hdr = nullptr;
    m__io__raw_te_hdr = nullptr;
    m_sections = nullptr;
    _read();
}

void uefi_te_t::_read() {
    m__raw_te_hdr = m__io->read_bytes(40);
    m__io__raw_te_hdr = std::unique_ptr<kaitai::kstream>(new kaitai::kstream(m__raw_te_hdr));
    m_te_hdr = std::unique_ptr<te_header_t>(new te_header_t(m__io__raw_te_hdr.get(), this, m__root));
    m_sections = std::unique_ptr<std::vector<std::unique_ptr<section_t>>>(new std::vector<std::unique_ptr<section_t>>());
    const int l_sections = te_hdr()->num_sections();
    for (int i = 0; i < l_sections; i++) {
        m_sections->push_back(std::move(std::unique_ptr<section_t>(new section_t(m__io, this, m__root))));
    }
}

uefi_te_t::~uefi_te_t() {
    _clean_up();
}

void uefi_te_t::_clean_up() {
}

uefi_te_t::te_header_t::te_header_t(kaitai::kstream* p__io, uefi_te_t* p__parent, uefi_te_t* p__root) : kaitai::kstruct(p__io) {
    m__parent = p__parent;
    m__root = p__root;
    m_data_dirs = nullptr;
    _read();
}

void uefi_te_t::te_header_t::_read() {
    m_magic = m__io->read_bytes(2);
    if (!(magic() == std::string("\x56\x5A", 2))) {
        throw kaitai::validation_not_equal_error<std::string>(std::string("\x56\x5A", 2), magic(), _io(), std::string("/types/te_header/seq/0"));
    }
    m_machine = static_cast<uefi_te_t::te_header_t::machine_type_t>(m__io->read_u2le());
    m_num_sections = m__io->read_u1();
    m_subsystem = static_cast<uefi_te_t::te_header_t::subsystem_enum_t>(m__io->read_u1());
    m_stripped_size = m__io->read_u2le();
    m_entry_point_addr = m__io->read_u4le();
    m_base_of_code = m__io->read_u4le();
    m_image_base = m__io->read_u8le();
    m_data_dirs = std::unique_ptr<header_data_dirs_t>(new header_data_dirs_t(m__io, this, m__root));
}

uefi_te_t::te_header_t::~te_header_t() {
    _clean_up();
}

void uefi_te_t::te_header_t::_clean_up() {
}

uefi_te_t::header_data_dirs_t::header_data_dirs_t(kaitai::kstream* p__io, uefi_te_t::te_header_t* p__parent, uefi_te_t* p__root) : kaitai::kstruct(p__io) {
    m__parent = p__parent;
    m__root = p__root;
    m_base_relocation_table = nullptr;
    m_debug = nullptr;
    _read();
}

void uefi_te_t::header_data_dirs_t::_read() {
    m_base_relocation_table = std::unique_ptr<data_dir_t>(new data_dir_t(m__io, this, m__root));
    m_debug = std::unique_ptr<data_dir_t>(new data_dir_t(m__io, this, m__root));
}

uefi_te_t::header_data_dirs_t::~header_data_dirs_t() {
    _clean_up();
}

void uefi_te_t::header_data_dirs_t::_clean_up() {
}

uefi_te_t::data_dir_t::data_dir_t(kaitai::kstream* p__io, uefi_te_t::header_data_dirs_t* p__parent, uefi_te_t* p__root) : kaitai::kstruct(p__io) {
    m__parent = p__parent;
    m__root = p__root;
    _read();
}

void uefi_te_t::data_dir_t::_read() {
    m_virtual_address = m__io->read_u4le();
    m_size = m__io->read_u4le();
}

uefi_te_t::data_dir_t::~data_dir_t() {
    _clean_up();
}

void uefi_te_t::data_dir_t::_clean_up() {
}

uefi_te_t::section_t::section_t(kaitai::kstream* p__io, uefi_te_t* p__parent, uefi_te_t* p__root) : kaitai::kstruct(p__io) {
    m__parent = p__parent;
    m__root = p__root;
    f_body = false;
    _read();
}

void uefi_te_t::section_t::_read() {
    m_name = kaitai::kstream::bytes_to_str(kaitai::kstream::bytes_strip_right(m__io->read_bytes(8), 0), std::string("UTF-8"));
    m_virtual_size = m__io->read_u4le();
    m_virtual_address = m__io->read_u4le();
    m_size_of_raw_data = m__io->read_u4le();
    m_pointer_to_raw_data = m__io->read_u4le();
    m_pointer_to_relocations = m__io->read_u4le();
    m_pointer_to_linenumbers = m__io->read_u4le();
    m_num_relocations = m__io->read_u2le();
    m_num_linenumbers = m__io->read_u2le();
    m_characteristics = m__io->read_u4le();
}

uefi_te_t::section_t::~section_t() {
    _clean_up();
}

void uefi_te_t::section_t::_clean_up() {
    if (f_body) {
    }
}

std::string uefi_te_t::section_t::body() {
    if (f_body)
        return m_body;
    std::streampos _pos = m__io->pos();
    m__io->seek(((pointer_to_raw_data() - _root()->te_hdr()->stripped_size()) + _root()->te_hdr()->_io()->size()));
    m_body = m__io->read_bytes(size_of_raw_data());
    m__io->seek(_pos);
    f_body = true;
    return m_body;
}