4.5 Pickles

4.5.1 poke Commands versus Poke Constructions

In Nomenclature we mentioned that poke, the program, implements a domain specific programming language called Poke, with a big p. In the examples so far we have already used the Poke language, somewhat extensively, while interacting with the program using the REPL.

For example, in:

(poke) 10 + 2
12

We are giving poke a Poke expression 10 + 2 to be evaluated. Once the expression is evaluated, the REPL prints the resulting value for us.

Similarly, when we define a variable or a type with var and type respectively, we are providing poke definitions to be evaluated. When we assign a value to a variable we are actually providing a Poke statement to be executed.

So the REPL accepts poke commands, some of which happen to be Poke expressions, definitions or statements. But we also have used dot-commands like .file or .info. These dot-commands are not part of the Poke programming language.

Every time we insert a line in the REPL and hit enter, poke recognizes the nature of the line, and then does the right thing. If the line is recognized as a Poke expression, for example, the Poke compiler is used to compile the statement into a routine, that is executed by the Poke Virtual Machine. The resulting value is then printed for the benefit of the user.

4.5.2 Poke Files

Poke programs are basically a collection of definitions and statements, which most often than not are stored in files, which avoids the need to type them again and again. By convention, we use the .pk file extension when naming files containing Poke programs.

Remember how we defined the foreground and background pixels for p.sbm in the REPL?

(poke) var bga = [255UB, 255UB, 255UB]
(poke) var fga = [255UB, 99UB, 71UB]

Where bga is a white pixel and fga is a tomato colored pixel. We could write these definitions in a file colors.pk like this:

var white = [255UB, 255UB, 255UB];
var tomato = [255UB, 99UB, 71UB];

Note that variable definitions in Poke are terminated by a semicolon (;) character, but we didn’t need to specify them when we issued the definitions in the REPL. This is because poke adds the trailing semicolon for us when it detects a Poke construction requiring it is introduced in the REPL.

Another difference is that Poke constructions can span for multiple lines, like in most programming languages. For example, we could have the following variable definition in a file:

var matrix = [[10, 20, 30],
              [40, 50, 60],
              [70, 80, 90]];

Once we have written our colors.pk file, how can we make poke aware of it? A possibility is to use the load construction:

(poke) load colors

Assuming a file named colors.pk exists in the current working directory, poke will load it and evaluate its contents. After this, we can use the colors:

(poke) tomato
[0xffUB,0x63UB,0x47UB]

Before, we would need to define these colors every time we would like to poke SBM files or, in fact, any RGB24 data. Now we just have to load the file colors.pk and use the variables defined there.

If you try to load a file whose name contains a dash character (-) you will get an error message:

(poke) load my-colors
<stdin>:1:8: error: syntax error, unexpected '-', expecting ';'
load my-colors;
       ^

This is because the argument to load is interpreted as a Poke identifier, and dash characters are not allowed in identifiers. To alleviate this problem, you can also specify the name of the file to load encoded in a string, line in:

(poke) load "my-colors.pk"

If you use this form of load, however, you are required to specify the complete name of the file, including the .pk file extension.

Since loading files is such a common operation, poke provides a dot-command .load that does auto-completion:

(poke) .load my-colors.pk

Which is equivalent to load "my-colors.pk".

Since load is part of the Poke language, it can also be used in Poke programs stored in files. We will explore this later.

4.5.3 Pickling Abstractions

In the last section we defined a couple of RGB colors white and tomato in a file called colors.pk. If we keep adding colors to the file, we may end with a nice collection of colors that are available to us at any time, by just loading the file.

Since there are many ways to understand the notion of “color”, and also many ways to implement these many notions, it would be better to be more precise and call our file rgb24.pk, since the notion of color we are using is of that RGB24. While are at it, let’s also rename the variables to reflect the fact they denote not just any kind of colors, but RGB24 colors:

var rgb24_white = [255UB, 255UB, 255UB];
var rgb24_tomato = [255UB, 99UB, 71UB];

At this point, we can also add the definitions of a couple of types to our rgb24.pk:

type RGB_Color_Beam = byte;
type RGB24_Color = RGB_Color_Beam[3];

var rgb24_white = [255UB, 255UB, 255UB];
var rgb24_tomato = [255UB, 99UB, 71UB];

Any time we want to manipulate RGB24 colors we can just load the file rgb24.pk and use these types and variables.

In poke parlance we call files like the above, that contain definitions of conceptually related entities, pickles. Pickles can be very simple, like the rgb24.pk sketched above, or fairly complicated like dwarf.pk.

It is common for pickles to load other pickles. For example, if we were to write a SBM pickle, we would load the RGB24 pickle from it:

load rgb24;

[... SBM definitions ...]

This way, when we load sbm from the repl, the dependencies get loaded as well.

GNU poke includes several already written pickles for commonly used file formats, and other domains. The load construction knows where these pickles are installed, so in order to load the pickle to manipulate ELF files, for example, all you have to do is to:

(poke) load elf

4.5.4 Exploring Pickles

As we have seen, a pickle provides Poke variables, types and functions related to some definite domain. Let’s say we are interested in editing an ELF file. We know that GNU poke comes with a pre-installed pickle named elf.pk. We can load it like this:

(poke) load elf

By convention, the entities provided by a pickle foo.pk are usually prefixed like:

Therefore, once the pickle is loaded we can use the .info dot-command in order to get a glimpse of the functionality provided by the pickle. See .info. Example:

(poke) .info variables elf
Name            Declared at
elf_stb_names   elf-common.pk:53
elf_stt_names   elf-common.pk:72
elf_stv_names   elf-common.pk:84

The above tells us that the elf.pk pickle provides these three variables. Types are way more interesting:

(poke) .info types Elf64
Name                 Declared at
Elf64_Ehdr           elf-64.pk:184
Elf64_Off            elf-64.pk:26
Elf64_SectionFlags   elf-64.pk:130
Elf64_Shdr           elf-64.pk:152
Elf64_RelInfo        elf-64.pk:36
Elf64_Chdr           elf-64.pk:85
Elf64_Rela           elf-64.pk:61
Elf64_Phdr           elf-64.pk:169
Elf64_Sxword         elf-64.pk:24
Elf64_Sym            elf-64.pk:71
Elf64_Rel            elf-64.pk:45
Elf64_File           elf-64.pk:207
Elf64_Addr           elf-64.pk:25
Elf64_Group          elf-64.pk:121
Elf64_Dyn            elf-64.pk:98
Elf64_Xword          elf-64.pk:23

We immediately notice the Elf64_File. Looks like what we would need to map in order to access the entire ELF data in some given IO space (like a file.) We can ask for more details about it:

(poke) .info type Elf64_File
Class:      struct
Name:       Elf64_File
Complete:   no
Fields:
  Name     Type
  ehdr     Elf64_Ehdr
  shdr     Elf64_Shdr[]
  phdr     Elf64_Phdr[]
Methods:
  Name                     Type
  get_section_name         (offset<Elf_Word,8>)string
  get_symbol_name          (Elf64_Shdr,,offset<Elf_Word,8>)string
  get_sections_by_name     (string)Elf64_Shdr[]
  get_sections_by_type     (Elf_Word)Elf64_Shdr[]
  section_name_p           (string)int
  get_string               (offset<Elf_Word,8>)string
  get_group_signature      (Elf64_Shdr)string
  get_group_signatures     ()string[]
  get_section_group        (string)Elf64_Shdr[]

We see that an Elf64_File has three fields, and the nature of these files: a header ehdr, an array of section headers shdr and an array of program headers phdr.

The type description above also lists the methods defined in the type. Looking at the method name and its type is very usually revealing enough to use it. For example, we see that get_sections_by_name returns an array of section headers, characterized by some given section name:

(poke) var elf = Elf64_File @ 0#B
(poke) elf.get_sections_by_name (".text")
[Elf64_Shdr {
  sh_name=0x1bU#B,
  sh_type=0x1U,
  sh_flags=#<ALLOC,EXECINSTR>,
  sh_addr=0x0UL#B,
  sh_offset=0x40UL#B,
  sh_size=0xbUL#B,
  sh_link=0x0U,
  sh_info=0x0U,
  sh_addralign=0x1UL,
  sh_entsize=0x0UL#b
}]

Ultimately, of course the best way to see what a pickle provides is to go read its source code.

4.5.5 Startup

When poke starts it loads the file .pokerc located in our home directory, if it exists. This initialization file contains poke commands, one per line.

If we wanted to get some Poke file loaded at startup, we could do it by adding a load command to our .pokerc. For example:

# My poke configuration - jemarch
[...]
.load ~/.poke.d/mydefs.pk