From 121209ac9702979d16b73340ec6a3a38adbe1b03 Mon Sep 17 00:00:00 2001
From: Andreas Baumann <mail@andreasbaumann.cc>
Date: Sun, 10 Oct 2021 19:44:33 +0200
Subject: added documenation from REQUIREMENTS to cc.wg (README.html) added a
 TODO file added an automatic build file with entr

---
 miniany/README.html  | 257 ++++++++++++++++++++++++++++++++++++++++++++++++---
 miniany/REQUIREMENTS | 142 ----------------------------
 miniany/TODOS        |   0
 miniany/autobuild.sh |  17 ++++
 miniany/cc.wg        | 191 ++++++++++++++++++++++++++++++++++----
 5 files changed, 435 insertions(+), 172 deletions(-)
 delete mode 100644 miniany/REQUIREMENTS
 create mode 100644 miniany/TODOS
 create mode 100755 miniany/autobuild.sh

diff --git a/miniany/README.html b/miniany/README.html
index 53eac72..7be5708 100644
--- a/miniany/README.html
+++ b/miniany/README.html
@@ -2,18 +2,18 @@
 <h2>Introduction</h2>
 <p>On the never-ending quest of a minimal system I found Swieros and C4 (the C compiler in 4 functions). Inspired and intrigued I started to implement my own.</p>
 <p>For abaos (a small operating system of mine, also in C) I cloned the minimal C library, so we can build a freestanding version of C4.</p>
-<p>C4 serves as a test whether my own CC is minimal enough and doesn't use silly functions. Additionally C4 as well as CC are compiled both in a (on Linux) hosted version and a freestanding version. We use a series of compilers like gcc, clang, tcc and pcc to make sure that we are not using silly C constructs.</p>
+<p>C4 serves as a test whether my own CC is minimal enough and doesn't use silly functions. Additionally C4 as well as CC are compiled both in a (on Linux) hosted version and a freestanding version. We use a series of compilers like gcc, clang, tcc and pcc to make sure that we are not using more silly C constructs.</p>
 <p>In order to be able to port easily we make almost no use of system calls, the ones we need are:</p>
 <ul>
-<li>brk: for malloc/free, change the start address of the heap segment of the process, if the OS only assigns a single static space, then brk results in a NOP.</li>
-<li>exit: terminate the process, return does not always work in all combinations (for instance with pcc on Linux). Can be a NOP, we don't require any trickery as <i>atext </i>and we don't use buffering anywhere (for instance flushing stdout on exit).</li>
-<li>read/write: read from stdin linearly, write to stdout linearly, this is essentially a model using an input and an output tape. Those two functions must really exist. This basically eliminates the need for a file system which we might not have during early bootstrapping.</li>
+<li><i>brk</i>: for malloc/free, change the start address of the heap segment of the process, if the OS only assigns a single static space, then brk results in a NOP.</li>
+<li><i>exit</i>: terminate the process, return does not always work in all combinations (for instance with pcc on Linux). Can be a NOP, we don't require any trickery as <i>atexit </i>and we don't use buffering anywhere (for instance flushing stdout on exit).</li>
+<li><i>read/write</i>: read from stdin linearly, write to stdout linearly, this is essentially a model using an input and an output tape. Those two functions must really exist. This basically eliminates the need for a file system which we might not have during early bootstrapping.</li>
 </ul>
 <p>Similarly we simplify the C language to not use certain features which can cause trouble when bootstrapping:</p>
 <ul>
-<li>variable arguments: though simple in principle (just some pointers into the stack if you use a stack for function parameters), it is not typesafe. And the only example in practice it's really heavily used for is in printf-like functions.</li>
-<li>preprocessor: it needs a filesystem, we take this outside of the compiler by feeding it an (eventually) concatenated list of *.c files.</li>
-<li>two types: int and char, so we can interpret memory as words or as bytes.</li>
+<li><i>variable </i><i>arguments</i>: though simple in principle (just some pointers into the stack if you use a stack for function parameters), it is not typesafe. And the only example in practice it's really heavily used for is in printf-like functions.</li>
+<li><i>preprocessor</i>: it needs a filesystem, we take this outside of the compiler by feeding it an (eventually) concatenated list of *.c files. Note: in the hosted environemt we (and glibc) can use as many preprocessor features as they want, they just don't have to get visible in our code.</li>
+<li><i>two </i><i>types</i>: int and char, so we can interpret memory as words or as bytes.</li>
 </ul>
 <h2>Local version of C4</h2>
 <p>The local version of C4 has the following adaoptions and extensions:</p>
@@ -29,11 +29,12 @@
 <li>BSD-style string functions like <i>strlcpy</i>, <i>strlcat</i></li>
 <li>strict C89 conformance, mainly use standard comment blocks, also removed some warnings</li>
 <li>some casts around malloc and memset to fit to non-void freestanding-libc</li>
-<li>converted printf to putstring/putint/putnl and some helper functions for error reporting like error()</li>
+<li>converted <i>print</i>f to <i>putstring/putint/putnl </i>and some helper functions for error reporting like error()</li>
 <li>removed all memory leaks</li>
-<li>de-POSIX-ified, no open/read/close, use getchar from stdin only (don't assume the existence of a file system), this also means we had to create sort of an old style tape-file with FS markers to separate the files piped to c4.</li>
+<li>de-POSIX-ified, no <i>open/read/close</i>, use <i>getchar </i>from stdin only (don't assume the existence of a file system), this also means we had to create sort of an old style tape-file with FS markers to separate the files piped to c4.</li>
 </ul>
-<p><i>Note: </i>only too late I discovered that there was a C5 version of the same compiler, which would maybe have served better as a basis.</p>
+<p>The reason for all those adaptions is to minimize the dependency on the host system and to be able to use <i>libc-freestanding.c</i>.</p>
+<h3><i>Note: </i>only too late I discovered that there was a C5 version of the same compiler, which would maybe have served better as a basis.</h3>
 <h2>Examples</h2>
 <h3>Running on the host system using the hosts C compiler</h3>
 <p>Compiled in either hosted (host libc) or freestanding (our own libc, currently IA-32 Linux kernel only syscalls):</p>
@@ -64,14 +65,248 @@ cat c4.c4 EOF c4.c EOF cc.c EOF hello.c | ./c4
 <p>We can also use <i>-s</i>, or <i>-d </i>on every level as follows:</p>
 <pre><code>cat cc.c EOF hello.c | ./c4 -d
 </code></pre>
+<h2>Features and Requirements</h2>
+<p>We have to careful what to put in a bootstrapping compiler, there is a tradeoff between</p>
+<ul>
+<li>costs to implement it in the language: code complexity and size must be kept at a minimun</li>
+<li>required features from the C runtime: for instance support of the whole Unicde characters</li>
+<li>required features from the operating system: for instance the requirement for a POSIX layer</li>
+<li>ugliness of the resulting code: it doesn't make sense to omit features which are somewhat hard to implement but can render readability of the code much better: best examples are the <i>switch/case </i>vs. <i>if/else </i>and the funny array indexing vs. structures in C4.</li>
+</ul>
+<p>So we collect some ideas here about features we add or do not add and why. We also collect here what their implications are, when we are implementing them.</p>
+<p>We also have to be careful what C4 can do for us and either add it there (but only if small enough) in order no to loose this test case.</p>
+<h3>Preprocessor for modularisation</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li><i>#include </i><i>&lt;filename.h&gt;</i>: requires a filename and thus an operating system with a filesystem early on, needs directory functions and open/close/read/write on files</li>
+<li>C4 ignores all prepocessor commands and treats them as comments till the end of line</li>
+</ul>
+<p>Alternative:</p>
+<ul>
+<li>have a <i>cat </i><i>*.c </i>concatenating all the source code</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>we could have sort of a special tape filesystem with a directory at the beginning and offset to allow the preprocessor to find the files for the early destination OS</li>
+<li>there is no reason not to use the preprocessor on the host</li>
+</ul>
+<h3>Preprocessor for conditional compilation</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>especially simple #ifdefs would be simple to do, conditional compilation in the same file is questionable, there is nothing wrong with including files like <i>linux_386.c</i>, <i>linux_x86_64.c</i></li>
+<li>they are not part of C, they are an addon (though they are nowadays sometimes treated as if they were part of C)</li>
+<li>C4 ignores all prepocessor commands and treats them as comments till the end of line</li>
+</ul>
+<p>Alternative:</p>
+<ul>
+<li>Use special files and concatenate them at the right place, e. g. <i>_start-stub.c </i>for tcc</li>
+</ul>
+<h3>Preprocessor for constant declarations</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>enum constants serve the same purpose, we prefer those</li>
+<li>even C89 can work with enum constants</li>
+</ul>
+<p>Caveats:</p>
+<ul>
+<li>C4 has only positive enums, we need negative ones for <i>EOF</i></li>
+<li>enum constants are signed integer, so we have to be careful when using them for chars</li>
+<li>we don't use enum for enumeration types</li>
+</ul>
+<h3>Variable Initializers</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>initializers of global and locals, not that important as we use C89 anyway, forcing us to separate declaration and usage of variables per scope</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>for example <i>symname[t]: </i>printing the symbol and not the number, requires static initializers for array of char* to make the code look nice. Of course we can also just initialize it in a piece of code.</li>
+</ul>
+<h3>Inline Assembly</h3>
+<p>Implementation status: yes</p>
+<p>Reasoning:</p>
+<ul>
+<li>somehow we have to communicate to the operating system, it's either inline assembly or external linking to assembly code</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>C4 has no inline assembly, we must add it there too</li>
+</ul>
+<p>Alternative:</p>
+<ul>
+<li>we could implement a linker and object format early on, we could use an external assembler as for instance <i>asm-i386 </i>(which understands a subset of <i>fasm</i>)</li>
+<li>we could implement an early dump format for symbols (function signatures mainly) and use them during linking</li>
+</ul>
+<p>Some general notes:</p>
+<p>GNU inline asm statement has become the de-facto standard (which is too complicated IMHO): I would require sort of a <i>.byte </i><i>0xX</i>X instruction only, for readablility maybe simple fasm-like syntax. We must be careful that our invention of an inline assembler can be mapped somehow to the GNU inline asm version, so that we can use that one on the host with gcc/clang/tcc/pcc..</p>
+<p>c.c in swieros (the c4 successor) has <i>asm(NOP)</i>, this is something we could implement easily. u.h contains an enum with opcodes (most likely doable or an easy architecture like the one in swieros, I doubt this works for Intel opcodes, but we should check if it works for our simplified Intel opcode subset).</p>
+<p>There should though be only one single point of information for opcodes per architecture, so asm gets sort of an inline string generator for the assembly output. Or we share a common C-file with enums for the opcodes and cat it to both the assembler and the compiler during the build (should not result in increaed code size, as those are enums).</p>
+<p>The asm(x) or asm(x,y) constructs can be mapped on the host compilers to asm __volatile__ .byte ugliness. In cc and c4 we can take the swieros approach. This should give us nice lowlevel inline assembly in a really simplified way (basically embedding bytes).</p>
+<p>Not having inline assembly means you need compilation units written and linked to the program in assembly, which - well - adds a linker and calling conventions, which might be too early in bootstrapping.</p>
+<h3>Object formats and linkers</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>requires file systems and linker formats like ELF</li>
+</ul>
+<p>Alternative:</p>
+<ul>
+<li>make compilation units from a bunch of source code, this results in bigger binaries, as we cannot share too much. This might be acceptable for early bootstrapping. Later on it would be nice to add a linker as an optional addon (which can be used outside of bootstrapping).</li>
+</ul>
+<h3>Forward declarations of function prototypes</h3>
+<p>Implementation status: yes (<b>TODO</b>)</p>
+<p>Reasoning:</p>
+<ul>
+<li>Recursive descent parsing requires forward function declarations.</li>
+</ul>
+<p>Caveats:</p>
+<ul>
+<li>Forward function declarations are not that easy to implement, because you have to generate a placeholder for the call address before you get the whole definition of the forwarded function (especially its entry address). Or we have to create sort of a temporary jump into a jump table (sort of a GOT) which we patch when we know the address of the implementation of the function. Either way we loose the 1-pass output of the generated code. Having a global table at one place scales easier, as we don't have to keep the whole generated code around just for patching (remember, we have tapes and memory, no seek of files).</li>
+<li>Must be added to c4 too, might not be that easy (<b>TODO</b>)</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>We could write a non-recursive parser using tables</li>
+</ul>
+<h3>Functions with variable arguments</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>not type-safe</li>
+<li>only used for (s)printfs string manipulation and output functions</li>
+<li>C4 doesn't implement variable arguments for defining functions, so we cannot bootstrap a freestanding version</li>
+</ul>
+<p>Requirements</p>
+<ul>
+<li>on the hosted Linux envorinment we need syscalls to <i>syscal</i>, <i>int </i><i>0x80</i>, etc.</li>
+<li>we need inline assembly to create to syscalls</li>
+</ul>
+<p>Alternative:</p>
+<ul>
+<li>create simple functions like <i>putchar</i>, <i>putint</i>, <i>putstring </i>(one per basic type) and some helpers like <i>putnl</i></li>
+<li>use <i>stlcat </i>and <i>stlcpy </i>should be enought to compose label names</li>
+<li>instead of a generic <i>syscall(..) </i>we can easily work around this by having <i>syscall1, </i><i>syscall2, </i><i>syscall3, </i><i>... </i>with a fixed number of parameters</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>we deviate from the C standard, printf just belongs to C</li>
+<li>printf is actually not hard to implement in a type-unsafe-way</li>
+<li>syscalls are variable arguments</li>
+</ul>
+<h3>FILE* and stderr</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>requires FILE * structures, requires various write channels from the operating system</li>
+<li>we can write error messages into the output stream as comment <i>; </i><i>ERROR </i><i>in </i><i>line </i><i>32, </i><i>pos </i><i>2: </i><i>generic </i><i>error </i>(<b>TODO</b>)</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>unbuffered FILE * is simple to implement</li>
+<li>almost all operating systems have the notion of stdout and stderr, we can set them to the same channel if an OS doesn't have them.</li>
+</ul>
+<h3>Typedefs</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>typedefs are syntactic sugar for <i>typedef </i><i>struct </i><i>T </i>as T, not strictly necessary and they don't make our code look too ugly.</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>portability without preprocessor could make use of <i>typedef </i><i>ulong64 </i><i>unsigned </i><i>long </i><i>int </i>and similar constructs</li>
+</ul>
+<h3>For-loops</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>unless we start optimizing (SIMD) there is no real benefit, for a generic 'for', a strict for i=0 to N, i++ is easier to optimize, when you have a grammatical construct to help recognizing it.</li>
+<li>the C for loop has a funny ending semicolon issue and you can add arbitraty code blocks into the three parts of the for, this is way too generic for a minimalistic language</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>for loops are not hard to implement</li>
+</ul>
+<h3>Passing arguments to main</h3>
+<p>Implementation status: yes</p>
+<p>Reasoning:</p>
+<ul>
+<li>this is just passing an <i>in</i>t (<i>argc</i>) and a <i>char** </i>(argv) from _start to main. This is also a convention shared with the operating system when it calls our process. Without that we would have a hard time to parametrize the calls to our process both on the host and in the target OS.</li>
+</ul>
+<p>Counter argument:</p>
+<ul>
+<li>Do really everything over the input stream, but this would feel a little bit too mainframe-ish.</li>
+</ul>
+<h3>bool</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>C89 has no boolean</li>
+<li>useful, but not strigtly necessary, we can live with <i>int </i>holding a boolean value</li>
+</ul>
+<h3>Union</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>in the compiler there is little benefit of compressing parts of e. g. ASTs into unions</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>for bootstrapping the operating system we might need unions (as well as packing and alignment)</li>
+</ul>
+<h3>Dangling else</h3>
+<p>Implementation status: no</p>
+<p>Reasoning:</p>
+<ul>
+<li>Don't allow non-blocked if/else, just avoid then dangling else problem, it's uninteresting and in C not as bad as in PASCAL-like languages, as the block markers are just one character big.</li>
+</ul>
+<h3>Register Allocation</h3>
+<p>Implementation status: yes</p>
+<p>Reasoning:</p>
+<ul>
+<li>Compared to a stack machine even the simplest register allocation algorithm produces much better code</li>
+</ul>
+<h3>Abstract Syntax Trees</h3>
+<p>Implementation status: yes</p>
+<p>Reasoning:</p>
+<ul>
+<li>Delaying code generation is essential when doing an assignment (rvalue must be evaluated before lvalue), in const folding (do no generate code as you don't know the expression is a constant but instead just compute the current value of the constant expression).</li>
+<li>Separate semantic operation <i>array </i><i>index </i><i>evaluation </i>from <i>definition </i><i>of </i><i>the </i><i>size </i><i>of </i><i>an </i><i>array </i>for instance with the '[' character (we do not want to react on the scanner symbol directly).</li>
+<li>When evaluating a boolean expression we don't know yet it's context (can be in a <i>if/while </i>condition, in which case we would geneate a conditional jump or whether in an assignment where we would store the value to the boolean variable).</li>
+</ul>
+<p>Caveats:</p>
+<ul>
+<li>Try to keep the scope of an AST as small as possible and as big as necessary (the output of the parser should not be the complete source code). Minimal is an expression and some context, maybe maximal is the scope of a function.</li>
+</ul>
+<p>Counter arguments:</p>
+<ul>
+<li>Readable AST code needs some trees and pointers to structs, but we want those anyway for better readable code (C4 for instance is not that readable in it's original array indexing version).</li>
+</ul>
+<h3>Builtin functions</h3>
+<p>Implementation status: yes</p>
+<p>Reasoning:</p>
+<ul>
+<li>Adding <i>putint/getchar </i>style of functions as elements of the language is tempting, as it allows early debugging and testing (as in PASCAL). The fundemental conflict here is betweet Bootstrapping is better with stdout and stdin in the language (no function calls, no linker, etc. needed). But later on we want those functions be part of a language library and not of the langyage itself.</li>
+</ul>
+<p>Caveats:</p>
+<ul>
+<li>Avoid code duplication (inline assembly in the compiler for the keyword implementation and with intline assembly in the language library). (<b>TODO</b>)</li>
+</ul>
 <h2>References</h2>
 <p>Compiler construction in general:</p>
 <ul>
 <li><i>&quot;Compiler </i><i>Construction&quot;</i>&quot;, Niklaus Wirth</li>
 <li><a href="https://github.com/DoctorWkt/acwj">https://github.com/DoctorWkt/acwj</a>: a nice series on building a C compiler, step by step with lots of good explanations</li>
-<li><a href="https://www.engr.mun.ca/%7Etheo/Misc/exp">https://www.engr.mun.ca/~theo/Misc/exp</a>_parsing.htm#climbing, <a href="https://en.wikipedia.org/wiki/Operator-precedence">https://en.wikipedia.org/wiki/Operator-precedence</a>_parser#Precedence_climbing_method</li>
 <li><a href="https://github.com/lotabout/write-a-C-interpreter/blob/master/tutorial/en/">https://github.com/lotabout/write-a-C-interpreter/blob/master/tutorial/en/</a>, tutorial based on C4 how to build a C interpreter, explains nicely details in C4.</li>
 </ul>
+<p>Some special compiler building topics:</p>
+<ul>
+<li><a href="https://www.engr.mun.ca/%7Etheo/Misc/exp">https://www.engr.mun.ca/~theo/Misc/exp</a>_parsing.htm#climbing, <a href="https://en.wikipedia.org/wiki/Operator-precedence">https://en.wikipedia.org/wiki/Operator-precedence</a>_parser#Precedence_climbing_method</li>
+<li><a href="https://en.wikipedia.org/wiki/Strahler">https://en.wikipedia.org/wiki/Strahler</a>_number: justification for register numbers for register alloation (<b>TODO: </b><b>clarify</b>)</li>
+</ul>
 <p>C4:</p>
 <ul>
 <li><a href="https://github.com/rswier/c4.git">https://github.com/rswier/c4.git</a>, <i>C4 </i><i>- </i><i>C </i><i>in </i><i>four </i><i>functions</i>, Robert Swierczek, minimalistic C compiler running on an emulator on the IR, inspiration for this project</li>
diff --git a/miniany/REQUIREMENTS b/miniany/REQUIREMENTS
deleted file mode 100644
index a6fce03..0000000
--- a/miniany/REQUIREMENTS
+++ /dev/null
@@ -1,142 +0,0 @@
-implementing:
-
-- userland
-  - argument passing to main function (argc, argv)
-- libc
-  - print_char
-    - requires a 3 parameter syscall to 80h (Linux)
-      - requires
-        - inline assembly
-
-not implementing:
-- libc
-  - variadic functions are not type-safe, do we need them?
-    - printf -> putint, putchar, etc.
-      - format string only, as replacement for puts
-      - vararg required in compiler
-      - not type-safe
-    - snprintf no option, strcat, strstr etc also not really
-    - newer formating functions and logging: strfmon, error, warn, syslog
-    - syscall
-  - puts
-    - requires stdout, which is a FILE structure
-  - print_char
-    - requires a 3 parameter syscall to 80h (Linux)
-      - requires
-        - either inline assembly
-        - linker and calling convention
-- preprocessor
-  - have a cat building up the required modules instead
-  - needs file operations (at least open, close, read)
-  - needs a file system on the host and the destination
-    (alternative: have a tape-like file system)
-- linker
-  - have compilation units needs a linker do build
-    an executable
-- symname[t] printing the symbol and not the number,
-  requires static initializers for array of char*
-- ASTs are basically only useful when you start to optimize,
-  till then you can use an intermediate format (as C4) does
-  and a stack machine. They also make the code easier readable.
-  For use they force the introduction of pointers, references and structs.
-  In expression parsing we see, that const folding already needs
-  an AST, because we should not emit code when still reading
-  a constant expression. It also seperates syntactical stuff like '['
-  from logical stuff like 'declaration of array size' and 'derefencing
-  a pointer'.
-- void * allowing to omit (char *) from and to for instance structs
-  in dynamic memory management
-- typedefs are just syntactic sugar, I use them mostly for 'struct T' -> 'T'
-- initializers of global and locals, not that important as we use C89 anyway,
-  forcing us to separate declaration and usage of variables per scope
-- unions, useful to safe space in AST, but not strictly necessary  
-- bool, useful, but not strigtly necessary
-- enums as constant replacement (instead of preprocessor), realy enum types
-  are not really useful.
-- forward struct definitions or typedefs (handy for Compiler structure), but..
-  and we can work around by not producing any loops (hopefully)
-- for loop: unless we start optimizing (SIMD) there is no real benefit
-  for a generic 'for', a strict for i=0 to N, i++ is easier to optimize, when
-  you have a grammatical construct to help recognizing it.
-- register number for register alloation
-  https://en.wikipedia.org/wiki/Strahler_number
-- volatile: we are not doing any optimizations for now, so volatile (as const)
-  can just be a ignored keyword.
-- c4 freestanding
-  - uses some casts, the malloc ones are actually good for clarification,
-    the ones in memset are not so useful (this is all because we don't
-    have 'void *')
-  - open/read/close is POSIX, we would prefer either C style file handling
-    (we have it in libc-freestanding.c or some stdin, stdout thingy)
-  - again printf and varargs, either use libc-freestanding.c or revert to
-    putint, putstring, putnl..
-    - if (tk == '(') next(); else { printf("%d: open paren expected\n", line); exit(-1); }
-      =>
-      error("open paren expected"); }
-    - printf("%d: compiler error tk=%d\n", line, tk); exit(-1);
-    - printf("could not malloc(%d) symbol area\n") => remove size, also map to error
-    - printf("read() returned %d\n", i); => dito
-    - we also print a non-sensical line, but we don't really care about this
-    - printf("%d: bad enum identifier %d\n", line, tk); he number 'tk' looks like
-      debug output here, so we drop it.
-      error1int is the other option (also choosen in other places)
-    - other cases translate by hand:
-      - case EXIT: /* putstring("exit("); putint(*sp); putstring(") cycle = "); putint(cycle); putnl(); */ return *sp;
-      - default: putstring("unknown instruction = "); putint(i); putstring("! cycle = "); putint(cycle); putnl(); return -1;
-  TODO:
-  - global char array declarations
-  - void parameter
-TODO:
-- avoid GNU-stype inline assembler (is far too complex), have more a
-  inline bytecode adder for explicit opcodes, e.g. nop -> .byte 0x90
-  - c.c in swieros (the c4 successor) has 'asm(NOP)', this is something we
-    could implement easily, preferably just as 'asm(0x90)'.
-    u.h contains an enum with opcodes (most likely doable or an easy architecture
-    like the one in swieros, I doubt this works for Intel opcodes).
-    There should though be only one single point of information for
-    opcodes per architecture, so asm gets sort of an inline string
-    generator for the assembly output. Or we share a common C-file with
-    enums for the opcodes and cat it to both the assembler and the compiler
-    during the build (should not result in increaed code size, as
-    those are enums).
-    the asm(x) or asm(x,y) constructs can be mapped on the host compilers
-    to asm __volatile__ .byte ugliness. In cc and c4 we can take the swieros
-    approach. This should give us nice lowlevel inline assembly in a really
-    simplified way (basically embedding bytes).
-    Not having inline assembly means you need compilation units written
-    and linked to the program in assembly, which - well - adds a linker
-    and calling conventions, which might be too early in bootstrapping.
-- asm-i386: device a new version which runs on c4 and is again freestanding   
-  - static: just ignore, we don't have a linker, otoh, just rewrite it whithout static,
-    vararg, etc.
-- c4.c: checkout c5-AST branch (darn, that one looks more promising to extend!)
-- cc.c: putint as a command in the language for early debugging (as in early Pascal),
-  points to a fundamental conflict: bootstrapping is better with stdout and stdin in
-  the language (no linker, no function calls etc. needed). OTOH we don't want to
-  have I/O as part of the language later, more be in the standard library.
-  Inline assembly in the generated code duplicates code with the putint in libc-freestanding.
-- error output is not on stderr, well, are we going to add stdout, stderr now
-  or do we write errors as sort of assembly comments?
-- AST debate:
-  - expressions really require an AST (just the A_ASSIGN itself with its reversed
-    order). IF, WHILE, etc. not, they can be in an AST or be endcoded directly (or if
-    not, what kind of optimizations do we loose?)
-    - the context of a boolean expression can be an if (in this case we would generate
-      direcly the jnXX instruction) and the far less often seen case of assigning it to
-      a variable. Knowing the contest would require an AST.
-  - AST should not be the output of whole programs, scoping is maybe better
-- don't allow non-blocked if/else, just avoid dangling else problems
-- for loops: they simply have a too general and weird semantic in C (missing
-  semicolon after the last triplet). IMHO a for loop makes sense for SIMD
-  operations only when we can use a stricter grammar to optimize certain
-  iterations.
-- c4: recursive descent parsing requires forward function declarations. Forward
-  function declarations are not that easy to implement, because you have to
-  generate a placeholder for the call address before you get the whole
-  definition of the forwarded function (especially its entry address).
-  Or we create sort of a temporary jump into a jump table (sort of a GOT) which
-  we patch when we know the address of the implementation of the function.
-  Having a global table at one place scales easier, as we don't have to keep
-  the whole generated code around just for patching (remember, we have tapes
-  and memory, no seek of files).
-  
diff --git a/miniany/TODOS b/miniany/TODOS
new file mode 100644
index 0000000..e69de29
diff --git a/miniany/autobuild.sh b/miniany/autobuild.sh
new file mode 100755
index 0000000..e174ff4
--- /dev/null
+++ b/miniany/autobuild.sh
@@ -0,0 +1,17 @@
+#!/bin/bash
+
+cat >/tmp/autobuild_tmp_$$.sh <<EOF
+#!/bin/bash
+
+echo "---"
+./build.sh cc tcc hosted d
+./cc < test1.c > test1.asm
+fasm test1.asm test1.bin
+../ecomp-c/emul -v test1.bin
+EOF
+chmod 775 /tmp/autobuild_tmp_$$.sh
+
+ls cc.c c4.c ../ecomp-c/emul.c ../ecomp-c/asm-i386.c test1.c | \
+	entr bash /tmp/autobuild_tmp_$$.sh
+rm -f /tmp/autobuild_tmp_$$.sh
+
diff --git a/miniany/cc.wg b/miniany/cc.wg
index 41f6c53..f09157a 100644
--- a/miniany/cc.wg
+++ b/miniany/cc.wg
@@ -28,17 +28,17 @@ WordGrinder dumpfile v3: this is a text file; diff me!
 .clipboard.cw: 1
 .clipboard.margin: 0
 .clipboard.viewmode: 1
-.clipboard.wordcount: 19
-.documents.1.co: 1
-.documents.1.cp: 26
-.documents.1.cw: 1
+.clipboard.wordcount: 6
+.documents.1.co: 9
+.documents.1.cp: 205
+.documents.1.cw: 21
 .documents.1.margin: 0
 .documents.1.name: "main"
 .documents.1.sticky_selection: false
 .documents.1.viewmode: 1
-.documents.1.wordcount: 937
+.documents.1.wordcount: 2863
 .fileformat: 8
-.findtext: "C4"
+.findtext: "Typedefs"
 .menu.accelerators.^@: "ZM"
 .menu.accelerators.^B: "SB"
 .menu.accelerators.BACKSPACE: "ZDPC"
@@ -146,22 +146,22 @@ WordGrinder dumpfile v3: this is a text file; diff me!
 .statusbar: true
 .current: 1
 #clipboard
-LB http://selfie.cs.uni-salzburg.at/: C* self-hosting C compiler (also emulator, hypervisor) for RISCV, inspiration for what makes up a minimal C language
+P TODO: add rest of REQUIREMENTS here!
 .
 #1
 H1 CC - a self-hosting, bootstrappable, minimal C compiler
 H2 Introduction
 P On the never-ending quest of a minimal system I found Swieros and C4 (the C compiler in 4 functions). Inspired and intrigued I started to implement my own.
 P For abaos (a small operating system of mine, also in C) I cloned the minimal C library, so we can build a freestanding version of C4.
-P C4 serves as a test whether my own CC is minimal enough and doesn't use silly functions. Additionally C4 as well as CC are compiled both in a (on Linux) hosted version and a freestanding version. We use a series of compilers like gcc, clang, tcc and pcc to make sure that we are not using silly C constructs.
+P C4 serves as a test whether my own CC is minimal enough and doesn't use silly functions. Additionally C4 as well as CC are compiled both in a (on Linux) hosted version and a freestanding version. We use a series of compilers like gcc, clang, tcc and pcc to make sure that we are not using more silly C constructs.
 P In order to be able to port easily we make almost no use of system calls, the ones we need are:
-LB brk: for malloc/free, change the start address of the heap segment of the process, if the OS only assigns a single static space, then brk results in a NOP.
-LB exit: terminate the process, return does not always work in all combinations (for instance with pcc on Linux). Can be a NOP, we don't require any trickery as atext and we don't use buffering anywhere (for instance flushing stdout on exit).
-LB read/write: read from stdin linearly, write to stdout linearly, this is essentially a model using an input and an output tape. Those two functions must really exist. This basically eliminates the need for a file system which we might not have during early bootstrapping.
+LB brk: for malloc/free, change the start address of the heap segment of the process, if the OS only assigns a single static space, then brk results in a NOP.
+LB exit: terminate the process, return does not always work in all combinations (for instance with pcc on Linux). Can be a NOP, we don't require any trickery as atexit and we don't use buffering anywhere (for instance flushing stdout on exit).
+LB read/write: read from stdin linearly, write to stdout linearly, this is essentially a model using an input and an output tape. Those two functions must really exist. This basically eliminates the need for a file system which we might not have during early bootstrapping.
 P Similarly we simplify the C language to not use certain features which can cause trouble when bootstrapping:
-LB variable arguments: though simple in principle (just some pointers into the stack if you use a stack for function parameters), it is not typesafe. And the only example in practice it's really heavily used for is in printf-like functions.
-LB preprocessor: it needs a filesystem, we take this outside of the compiler by feeding it an (eventually) concatenated list of *.c files.
-LB two types: int and char, so we can interpret memory as words or as bytes.
+LB variable arguments: though simple in principle (just some pointers into the stack if you use a stack for function parameters), it is not typesafe. And the only example in practice it's really heavily used for is in printf-like functions.
+LB preprocessor: it needs a filesystem, we take this outside of the compiler by feeding it an (eventually) concatenated list of *.c files. Note: in the hosted environemt we (and glibc) can use as many preprocessor features as they want, they just don't have to get visible in our code.
+LB two types: int and char, so we can interpret memory as words or as bytes.
 H2 Local version of C4
 P The local version of C4 has the following adaoptions and extensions:
 LB switch statement from the switch-and-structs branch, adapted c4 itself to use switch statements instead of if's (as in the switch-and-structs branch)
@@ -175,10 +175,11 @@ LB some simplified functions for printing like putstring, putint, putnl rep
 LB BSD-style string functions like strlcpy, strlcat
 LB strict C89 conformance, mainly use standard comment blocks, also removed some warnings
 LB some casts around malloc and memset to fit to non-void freestanding-libc
-LB converted printf to putstring/putint/putnl and some helper functions for error reporting like error()
+LB converted printf to putstring/putint/putnl and some helper functions for error reporting like error()
 LB removed all memory leaks
-LB de-POSIX-ified, no open/read/close, use getchar from stdin only (don't assume the existence of a file system), this also means we had to create sort of an old style tape-file with FS markers to separate the files piped to c4.
-P Note: only too late I discovered that there was a C5 version of the same compiler, which would maybe have served better as a basis.
+LB de-POSIX-ified, no open/read/close, use getchar from stdin only (don't assume the existence of a file system), this also means we had to create sort of an old style tape-file with FS markers to separate the files piped to c4.
+P The reason for all those adaptions is to minimize the dependency on the host system and to be able to use libc-freestanding.c.
+H3 Note: only too late I discovered that there was a C5 version of the same compiler, which would maybe have served better as a basis.
 H2 Examples
 H3 Running on the host system using the hosts C compiler
 P Compiled in either hosted (host libc) or freestanding (our own libc, currently IA-32 Linux kernel only syscalls):
@@ -203,12 +204,164 @@ PRE cat c4.c4 EOF c4.c EOF cc.c EOF hello.c | ./c4
 P EOF contains the traditional FS (file separator) character in the ASCII character set. Every time c4 is invoked it reads exacly one input file up to the first FS character (or stops at the end of stdin).
 P We can also use -s, or -d on every level as follows:
 PRE cat cc.c EOF hello.c | ./c4 -d
-H2 References
+H2 Features and Requirements
+P We have to careful what to put in a bootstrapping compiler, there is a tradeoff between
+LB costs to implement it in the language: code complexity and size must be kept at a minimun
+LB required features from the C runtime: for instance support of the whole Unicde characters
+LB required features from the operating system: for instance the requirement for a POSIX layer
+LB ugliness of the resulting code: it doesn't make sense to omit features which are somewhat hard to implement but can render readability of the code much better: best examples are the switch/case vs. if/else and the funny array indexing vs. structures in C4.
+P So we collect some ideas here about features we add or do not add and why. We also collect here what their implications are, when we are implementing them.
+P We also have to be careful what C4 can do for us and either add it there (but only if small enough) in order no to loose this test case.
+H3 Preprocessor for modularisation
+P Implementation status: no
+P Reasoning:
+LB #include <filename.h>: requires a filename and thus an operating system with a filesystem early on, needs directory functions and open/close/read/write on files
+LB C4 ignores all prepocessor commands and treats them as comments till the end of line
+P Alternative:
+LB have a cat *.c concatenating all the source code
+P Counter arguments:
+LB we could have sort of a special tape filesystem with a directory at the beginning and offset to allow the preprocessor to find the files for the early destination OS
+LB there is no reason not to use the preprocessor on the host
+H3 Preprocessor for conditional compilation
+P Implementation status: no
+P Reasoning:
+LB especially simple #ifdefs would be simple to do, conditional compilation in the same file is questionable, there is nothing wrong with including files like linux_386.c, linux_x86_64.c
+LB they are not part of C, they are an addon (though they are nowadays sometimes treated as if they were part of C)
+LB C4 ignores all prepocessor commands and treats them as comments till the end of line
+P Alternative:
+LB Use special files and concatenate them at the right place, e. g. _start-stub.c for tcc
+H3 Preprocessor for constant declarations
+P Implementation status: no
+P Reasoning:
+LB enum constants serve the same purpose, we prefer those
+LB even C89 can work with enum constants
+P Caveats:
+LB C4 has only positive enums, we need negative ones for EOF
+LB enum constants are signed integer, so we have to be careful when using them for chars
+LB we don't use enum for enumeration types
+H3 Variable Initializers
+P Implementation status: no
+P Reasoning:
+P - initializers of global and locals, not that important as we use C89 anyway, forcing us to separate declaration and usage of variables per scope
+P Counter arguments:
+P - for example symname[t]: printing the symbol and not the number, requires static initializers for array of char* to make the code look nice. Of course we can also just initialize it in a piece of code.
+H3 Inline Assembly
+P Implementation status: yes
+P Reasoning:
+LB somehow we have to communicate to the operating system, it's either inline assembly or external linking to assembly code
+P Counter arguments:
+LB C4 has no inline assembly, we must add it there too
+P Alternative:
+LB we could implement a linker and object format early on, we could use an external assembler as for instance asm-i386 (which understands a subset of fasm)
+LB we could implement an early dump format for symbols (function signatures mainly) and use them during linking
+P Some general notes:
+P GNU inline asm statement has become the de-facto standard (which is too complicated IMHO): I would require sort of a .byte 0xXX instruction only, for readablility maybe simple fasm-like syntax. We must be careful that our invention of an inline assembler can be mapped somehow to the GNU inline asm version, so that we can use that one on the host with gcc/clang/tcc/pcc..
+P c.c in swieros (the c4 successor) has asm(NOP), this is something we could implement easily. u.h contains an enum with opcodes (most likely doable or an easy architecture like the one in swieros, I doubt this works for Intel opcodes, but we should check if it works for our simplified Intel opcode subset).
+P There should though be only one single point of information for opcodes per architecture, so asm gets sort of an inline string generator for the assembly output. Or we share a common C-file with enums for the opcodes and cat it to both the assembler and the compiler during the build (should not result in increaed code size, as those are enums).
+P The asm(x) or asm(x,y) constructs can be mapped on the host compilers to asm __volatile__ .byte ugliness. In cc and c4 we can take the swieros approach. This should give us nice lowlevel inline assembly in a really simplified way (basically embedding bytes).
+P Not having inline assembly means you need compilation units written and linked to the program in assembly, which - well - adds a linker and calling conventions, which might be too early in bootstrapping.
+H3 Object formats and linkers
+P Implementation status: no
+P Reasoning:
+LB requires file systems and linker formats like ELF
+P Alternative:
+LB make compilation units from a bunch of source code, this results in bigger binaries, as we cannot share too much. This might be acceptable for early bootstrapping. Later on it would be nice to add a linker as an optional addon (which can be used outside of bootstrapping).
+H3 Forward declarations of function prototypes
+P Implementation status: yes (TODO)
+P Reasoning:
+LB Recursive descent parsing requires forward function declarations.
+P Caveats:
+LB Forward function declarations are not that easy to implement, because you have to generate a placeholder for the call address before you get the whole definition of the forwarded function (especially its entry address). Or we have to create sort of a temporary jump into a jump table (sort of a GOT) which we patch when we know the address of the implementation of the function. Either way we loose the 1-pass output of the generated code. Having a global table at one place scales easier, as we don't have to keep the whole generated code around just for patching (remember, we have tapes and memory, no seek of files).
+LB Must be added to c4 too, might not be that easy (TODO)
+P Counter arguments:
+LB We could write a non-recursive parser using tables
+H3 Functions with variable arguments
+P Implementation status: no
+P Reasoning:
+LB not type-safe
+LB only used for (s)printfs string manipulation and output functions
+LB C4 doesn't implement variable arguments for defining functions, so we cannot bootstrap a freestanding version
+P Requirements
+LB on the hosted Linux envorinment we need syscalls to syscal, int 0x80, etc.
+LB we need inline assembly to create to syscalls
+P Alternative:
+LB create simple functions like putchar, putint, putstring (one per basic type) and some helpers like putnl
+LB use stlcat and stlcpy should be enought to compose label names
+LB instead of a generic syscall(..) we can easily work around this by having syscall1, syscall2, syscall3, ... with a fixed number of parameters
+P Counter arguments:
+LB we deviate from the C standard, printf just belongs to C
+LB printf is actually not hard to implement in a type-unsafe-way
+LB syscalls are variable arguments
+H3 FILE* and stderr
+P Implementation status: no
+P Reasoning:
+LB requires FILE * structures, requires various write channels from the operating system
+LB we can write error messages into the output stream as comment ; ERROR in line 32, pos 2: generic error (TODO)
+P Counter arguments:
+LB unbuffered FILE * is simple to implement
+LB almost all operating systems have the notion of stdout and stderr, we can set them to the same channel if an OS doesn't have them.
+H3 Typedefs
+P Implementation status: no
+P Reasoning:
+LB typedefs are syntactic sugar for typedef struct T as T, not strictly necessary and they don't make our code look too ugly.
+P Counter arguments:
+LB portability without preprocessor could make use of typedef ulong64 unsigned long int and similar constructs
+H3 For-loops
+P Implementation status: no
+P Reasoning:
+LB unless we start optimizing (SIMD) there is no real benefit, for a generic 'for', a strict for i=0 to N, i++ is easier to optimize, when you have a grammatical construct to help recognizing it.
+LB the C for loop has a funny ending semicolon issue and you can add arbitraty code blocks into the three parts of the for, this is way too generic for a minimalistic language
+P Counter arguments:
+LB for loops are not hard to implement
+H3 Passing arguments to main
+P Implementation status: yes
+P Reasoning:
+LB this is just passing an int (argc) and a char** (argv) from _start to main. This is also a convention shared with the operating system when it calls our process. Without that we would have a hard time to parametrize the calls to our process both on the host and in the target OS.
+P Counter argument:
+LB Do really everything over the input stream, but this would feel a little bit too mainframe-ish.
+H3 bool
+P Implementation status: no
+P Reasoning:
+LB C89 has no boolean
+LB useful, but not strigtly necessary, we can live with int holding a boolean value
+H3 Union
+P Implementation status: no
+P Reasoning:
+LB in the compiler there is little benefit of compressing parts of e. g. ASTs into unions
+P Counter arguments:
+LB for bootstrapping the operating system we might need unions (as well as packing and alignment)
+H3 Dangling else
+P Implementation status: no
+P Reasoning:
+LB Don't allow non-blocked if/else, just avoid then dangling else problem, it's uninteresting and in C not as bad as in PASCAL-like languages, as the block markers are just one character big.
+H3 Register Allocation
+P Implementation status: yes
+P Reasoning:
+LB Compared to a stack machine even the simplest register allocation algorithm produces much better code
+H3 Abstract Syntax Trees
+P Implementation status: yes
+P Reasoning:
+LB Delaying code generation is essential when doing an assignment (rvalue must be evaluated before lvalue), in const folding (do no generate code as you don't know the expression is a constant but instead just compute the current value of the constant expression).
+LB Separate semantic operation array index evaluation from definition of the size of an array for instance with the '[' character (we do not want to react on the scanner symbol directly).
+LB When evaluating a boolean expression we don't know yet it's context (can be in a if/while condition, in which case we would geneate a conditional jump or whether in an assignment where we would store the value to the boolean variable).
+P Caveats:
+LB Try to keep the scope of an AST as small as possible and as big as necessary (the output of the parser should not be the complete source code). Minimal is an expression and some context, maybe maximal is the scope of a function.
+P Counter arguments:
+LB Readable AST code needs some trees and pointers to structs, but we want those anyway for better readable code (C4 for instance is not that readable in it's original array indexing version).
+H3 Builtin functions
+P Implementation status: yes
+P Reasoning:
+LB Adding putint/getchar style of functions as elements of the language is tempting, as it allows early debugging and testing (as in PASCAL). The fundemental conflict here is betweet Bootstrapping is better with stdout and stdin in the language (no function calls, no linker, etc. needed). But later on we want those functions be part of a language library and not of the langyage itself.
+P Caveats:
+LB Avoid code duplication (inline assembly in the compiler for the keyword implementation and with intline assembly in the language library). (TODO)
+H2 References
 P Compiler construction in general:
 LB "Compiler Construction"", Niklaus Wirth
 LB https://github.com/DoctorWkt/acwj: a nice series on building a C compiler, step by step with lots of good explanations
-LB https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm#climbing, https://en.wikipedia.org/wiki/Operator-precedence_parser#Precedence_climbing_method
 LB https://github.com/lotabout/write-a-C-interpreter/blob/master/tutorial/en/, tutorial based on C4 how to build a C interpreter, explains nicely details in C4.
+P Some special compiler building topics:
+LB https://www.engr.mun.ca/~theo/Misc/exp_parsing.htm#climbing, https://en.wikipedia.org/wiki/Operator-precedence_parser#Precedence_climbing_method
+LB https://en.wikipedia.org/wiki/Strahler_number: justification for register numbers for register alloation (TODO: clarify)
 P C4:
 LB https://github.com/rswier/c4.git, C4 - C in four functions, Robert Swierczek, minimalistic C compiler running on an emulator on the IR, inspiration for this project
 LB https://github.com/rswier/c4/blob/switch-and-structs/c4.c, c4 adaptions to provide switch and structs
-- 
cgit v1.2.3-54-g00ecf