Using FramerD From C
Ken Haase

Quick ref: [Function name morphology]  [Lisp pointers]  [Exception Handling]  [Unicode Representation]  [Input from strings]  [Output to strings]  [Lisp Types]  (esp. immediates, strings, composites) [Evaluator Types]  [Evaluator Functions]  [Implementing Modules]  [Directory Structure]  [Source Files


The kernel of FramerD is written in ANSI C and one way of using FramerD is by writing C or C++ code and using the FramerD libraries in either static or dynamic form. This provisional document describes programming with these libraries and also describes the structure of the FramerD source tree.

Programming with FramerD

In order to program using FramerD from C, you need to know what compile flags to use and where to find headers and include files. The script fdxs provides this information in the form of command line strings which can be passed directly to compilers and linkers. For these purposes, there are three forms of the fdxs commands:

fdxs cflagsreturns the command line options to use for compiling programs which use the FramerD libraries. This includes options specifying where the FramerD include files live as well as other flags such as (on some platforms) -fshort-enums.
fdxs libsreturns the arguments for linking against the FramerD libraries, including the -L argument indicating where the libraries live and individual -llib arguments for the various FramerD libraries themselves.
fdxs modulesreturns the directory where FramerD modules should be installed.

The fdxs command can be used to write portable makefiles by using the backquoted shell escape to execute an fdxs and include its results in the compilation commands. For instance, here is the makefile for the xframerd module:

CFLAGS = `fdxs cflags` `libwww-config --cflags` $(XCFLAGS)

all:  w3fd.so

tidy:
	rm -f w3fd.o *~
clean: tidy
	rm -f w3fd.so

w3fd.so: w3fd.o makefile
	ld -shared -o $@ w3fd.o `libwww-config --libs` `fdxs libs`

install: w3fd.so
	cp -f w3fd.so `fdxs modules`

which uses `fdxs cflags` to get the C compiler command line, `fdxs libs` to get the linker command line arguments, and `fdxs modules` to get the modules directory.

Function name morphology

There are a few heuristics for determining the semantics of functions and macros based on their names. First, nearly all library functions and macros start with the prefix fd_ or FD_; if the preprocessor variable FD_SOURCE is defined to be 1 before the FramerD include files are referenced, some aliases without the fd_ prefix will be defined.

Function names starting with _fd_ are internal library functions typically used by preprocessor macros, and should not be directly called by user code. Most functions in the library should be threadsafe, but some functions have variants with the suffix _nolock indicating that they are not threadsafe.

Case is used to distinguish "well-behaved type-checking" functions from riskier functions. Lower case functions (or preprocessor macros which can be called as functions) typically do type checking (when handed LISP pointers) and obey standard conventions for interaction with the garbage collector (if they hold onto an argument they will incref it and if the return a lisp pointer it will need to be decrefed unless further returned. Some functions have upper case names (and are often implemented as macros or inline functions) and make no such guarantees. Other uppercase identifiers act as semi-syntactic elements, rather than as functions, and these are always implemented by C preprocessor macros (though they may have helper functions, generally starting with _fd_. These are mostly used in the exception handling system.

The FramerD libraries are divided into five core collections of functions:

dtype
provides utility functions, the core LISP data types used by FramerD, and binary DType I/O allowing these data types transmission and storage;
framerd
provides the basic object database, the associative indices used to do fast searches for objects, the core of the FDScript evaluator (including threading and the module and security system), and basic procedures for accessing frames and making simple inferences;
scheme
provides much of the R4RS scheme functionality, including the full arithmetic system, string and character operations, and I/O primitives; currently, these functions are lumped into the fdscript library from the developer's point of view;
fdscript
provides FDScript's special extensions, including more advanced FramerD access and maintenance functions, operating system access, the PRINTOUT family of functions, imports from Common LISP (including sequence functions, DOLIST, DOTIMES, etc.
fdtext
provides FDScript's text analysis tools, including a powerful and extensible textual pattern matcher, HTML and XML generation and parsing (including XML namespace handling), MIME and RFC-822 parsing, MD5 hashing, the Porter stemmer, and a simple proper name extractor.

All externally available functions are prefixed by a preprocessor macro providing information to the linker. In the core FramerD code, these macros are:

DTYPES_EXPORT, FRAMERD_EXPORT, FDSCRIPT_EXPORT, FDTEXT_EXPORT, EXPORTED

LISP Pointers: fd_lisp

This is a typed pointer. In the standard implementation, this is a struct of type struct FD_LISP_PTR consisting of a type tag (of enumerated type fd_lisp_type stored in the type field) and a data pointer (of union type fd_lisp_data stored in the ->data field). The type of a pointer can be directly tested with the macro FD_PRIM_TYPEP(x,type) where x is the lisp pointer and type is one of the values of fd_lisp_type. The data portion can be extracted with the function FD_PTR_DATA(x,field) where field indicates a particular kind of data.

Garbage Collection  FramerD uses a reference counting garbage collector, the important external functions/macros are:

The contract between fd_copy_lisp and fd_decref is that if an object y is created by fd_copy_lisp(x), it will be unaffected by a subsequent fd_decref(x). Nearly all normal FramerD library functions and preprocessor macros which take lisp pointers as arguments will incref those pointers if they will hang onto them; likewise nearly all functions returning LISP pointers will implicitly incref their argument. Functions and macros which do not do this will have either uppercase names or begin with an underscore "_", indicating that the function is an internal function.

Comparing Objects  Lisp objects can also be compared with one another:

Functions and macros for generating or accessing particular types of objects are shown below.

Printing LISP objects

fd_object_to_string(object)
Returns a string containing a printed representation for object.
fd_parse_lisp_from_stream(stream)
Returns an object generated from a printed representation read from stream (of type FILE *).
fd_parse_lisp_from_string
Returns an object generated from a printed representation stored in string
fd_print_dtype(object,file stream)
Emits a printed representation of an object onto file stream.
fd_print_dtype_to_stdout(object)
Writes a printed representation of object to the standard output, followed by a carriage return.
fd_print_dtype_to_string(object,string stream)
Writes a printed representation of object to a string data structure.
fd_read_dtype_from_file(filestring)
fd_str2lisp(string)
Converts a string containing a printed representation into a lisp object.
fd_arg2lisp(string)
Returns an object based on the printed representation in string. It is designed for parsing command line arguments as lisp objects and favors returning LISP strings. In particular, only arguments beginning with a colon are interpreted as symbols.

Reading and Writing DTypes

fd_dread_dtype(data_buffer)
Takes a data buffer and returns the object specified by the DType representation within it.
fd_dtype_compare(object,stream)
Returns 1 if the stream specifies a DTYPE representation for OBJECT.
fd_dwrite_dtype(object,data_buffer)
Writes a dtype representation for an object into a data buffer.
fd_fignore_dtype(stream)
Skips over one dtype representation on stream.
fd_fread_dtype(filestream)
Reads a dtype representation from a binary filestream, returning the corresponding object.
fd_fwrite_dtype(object,filestream)
Writes a DType representation of object to a filestream.
fd_write_dtype_to_file(object,filestring)
Writes a dtype representation of an object to the specified file.

Exception Handling

The type fd_exception is used by FramerD's exception handling system, this is a pointer to a string (char *) describing an error. The functions for dealing with errors can be found in src/os/except.c. The key functions for signalling errors are

Catching exceptions is done with macros that make C look a little like LISP. The basic schemas are:

{
  WITH_HANDLING {
    ... do some stuff ... }
  ON_EXCEPTION {
    ... do stuff when an exception gets raised ... }
  END_HANDLING
}

for catching exceptions and

{
  UNWIND_PROTECT {
    ... do some stuff ... }
  ON_UNWIND {
    ... do this even if an exception is raised ... }
  END_UNWIND
}

to ensure that cleanup code gets run despite exceptions. In either case, you can use the function fd_theException() to get the exception which was signalled, fd_exception_details() to get the details string for the exception, and fd_exception_object() to get whatever LISP object was associated with the acception (or FD_VOID if there wasn't any).

The exception handliing or protection code can call fd_reraise() to pass on the current exception, or fd_clear_exception() to clear the current exception. The protection code will automatically reraise the exception unless it has been cleared.

Some Standard Exceptions

fd_Out_Of_Memory
An exception signalled when a system call to malloc() has failed
fd_Parse_Error
An exception signalled when the parser for LISP expressions encounters an unexpected token
fd_Unexpected_EOD
An exception signalled when a data buffer ends unexpectedly.
fd_Unexpected_EOF
An exception signalled when a data stream ends unexpectedly.
fd_Unknown_Record_Type
An exception signalled when the system tries to operate on an unknown record type.
fd_Unknown_Type_Code
An exception signalled when when an unknown DTYPE data code is read. It generally occurs when mis-aligned data has been written out, which none of the built-in procedures should ever do.

Unicode

FramerD uses UTF-8 as its internal coding for Unicode strings. UTF-8 encodes ASCII characters using a single byte and encodes other Unicode code points using a series of non-ASCII (decimal value > 255) characters. Most strings in FramerD are UTF-8 strings and are typed as either fd_utf8_string or (more commonly) a pointer to an array of type fd_u8char (which is just an unsigned character).

Translations from external character sets to UTF-8 is managed by FramerD using structures of type struct FD_TEXT_ENCODING. The function fd_get_encoding(char *) returns a particular named encoding, e.g. fd_get_encoding("LATIN-1") returns the LATIN-1 encoding for Western European characters.

The function fd_make_utf8(char *start,char *end,struct FD_TEXT_ENCODING *e) returns a newly malloc'd UTF-8 string based on the characters between start and end based on the encoding e. If end is NULL, the entire string start is used; if e is NULL, the default encoding is used.

The function fd_convert_utf8(u8char *start,int len,struct FD_TEXT_ENCODING *e,int *sizep) does the opposite conversion, taking a UTF-8 string and producing a newly malloc'd string of characters using encoding e. If a given character cannot be represented in this encoding, an error is signalled. The length of the resulting string is stored in sizep if it is non-NULL (this is especially important if the generated string contains NULs ('\0') which will mislead the C libraries strlen.

The function fd_localize_utf8(u8char *start,struct FD_TEXT_ENCODING *e) does the opposite conversion, taking a UTF-8 string and producing a newly malloc'd string of characters using encoding e. If a given character cannot be represented in this encoding, an error is signalled.

FramerD provides a structure called an FD_XFILE (which deals with alien characters) for doing I/O using character sets. The function fd_get_xfile(FILE *f,struct FD_TEXT_ENCODING *e) returns an FD_XFILE structure (malloc'ing it if neccessary) which reads its input or writes its output using f together with the encoding e. This structure should be freed with the function fd_free_xfile.

The functions for directly using XFILEs are:

The FramerD internationalization library also provides a layer for using stdio FILE * pointers directly, without the intervening XFILEs. The function fd_set_file_encoding(FILE *f,char *name) associates the encoding named name with the file f. After this association, the following functions provide automatic encoding and decoding:

The encoding assigned by fd_set_file_encoding can be retrieved by fd_get_file_encoding.

In addition to file specific and XFILE encodings, there are three special global encodings: the console encoding is used for output to the console (stdin/stdout/stderr); the system encoding is used to convert strings before and after system or library calls, for instance filenames; finally, the default encoding is used whenever neither of these encodings are relevant and no other encoding is explicity provided.

The function fd_set_default_encoding(struct FD_TEXT_ENCODING *e) sets the default encoding and the function fd_get_default_encoding() retrieves it. Similarly, for the system encoding, the function fd_set_system_encoding(struct FD_TEXT_ENCODING *e) sets the default encoding and the function fd_get_system_encoding() retrieves it. The console encoding can be modified by applying the functions above

FramerD normally comes with built-in encodings for ASCII, LATIN-[1-9], ISO-8859/[1-15], UTF-8, UCS-2, and KOI-8. In addition, the function fd_load_encoding(char *name,char *file) loads a named encoding from an external file. It is able to interpret both the "charset mapping files" provided by the Unicode Consortium (ftp://ftp.unicode.org/Public/MAPPINGS/) and the native character set encoding used by Linux and some other operating systems. Under Linux, these mappings usually live in the /usr/share/i18n/charmaps/ directory.

The environment variable ENCODINGS_PATH can refer to a number of directories where FramerD automatically searches for encodings in this format. In this search, encoding names are canonicalized by removal of of punctuation characters and upper-casing of alphabetic characters.

The following program translates the standard input to the standard output, doing character translations automatically:

int main()
{
  struct FD_TEXT_ENCODING *in, *out; int c;
  in=fd_get_encoding(argv[1]); out=fd_get_encoding(argv[2]);
  fd_set_file_encoding(stdin,in); fd_set_file_encoding(stdout,out);
  while ((c=fd_fgetc(stdin)) >= 0) fd_fputc(c,stdout);
  exit(0);
}

Composed Characters

Some unicode characters are combining characters which work by modifying preceding or succeeding characters with (for instance) diacritical or other marks. This leads to a potential for ambiguity because some modified characters have two representations: a direct representation as a single unicode character and a combined representation consisting of the unmodified character and a subsequent modifier. The function fd_decompose_char(unsigned int ch) returns a UTF-8 string encoding ch as a sequence of base character plus modifiers; the function fd_recompose_char(fd_u8char *s) returns the single unicode character (if it exists) represented by the character and modifiers in the UTF-8 string s It returns -1 if the string does not describe a combined character with a single character equivalent.

The standard I/O functions (such as fd_fgetc) do not normally interpret combined characters, but return base and combining characters separately. The functions above are designed to be used when programs need to interpret such combined characters.

String Streams

String streams are for generating strings programmatically and are used extensively by FDScript and especially its text processing utilities. String streams grow automatically as output is received. A string stream is initialized by the macro FD_INITIALIZE_STRING_STREAM(ssp,isize) where ssp is a pointer to a string stream and isize is the initial number of bytes allocated to its buffer. This buffer is kept in the ->ptr field of the FD_STRING_STREAM (of type fd_u8char *). The current capacity of this buffer is an int in the limit and the current actual size is in the size field. These sizes are in bytes and the number of represented unicode characters may differ, since some Unicode characters take more than one byte of UTF-8 encoding.

As indicated by the type of the buffer, string streams use UTF-8 to represent Unicode output. The functions for doing output to string streams are:

For example, the following procedure returns a string based on a character range:

u8char *enumerate_char_range(int bot,int top)
{
  /* Return a string of characters from bot to top */
  struct FD_STRING_STREAM s; int i=bot;
  FD_INITIALIZE_STRING_STREAM(&s,(top-bot)*2);
  while (i < top) {fd_sputc(&s,i); i++;}
  return s.ptr;
}

The macro FD_INITIALIZE_FIXED_STRING_STREAM initializes a string stream of fixed size; output beyond the specified size is a no-op. It takes three arguments: a pointer to the string stream to initialize, the size of the buffer, and a pointer to the buffer (which is typically on the stack). We could use this to implement a limited version of the procedure above:

u8char *enumerate_char_range(int bot,int top)
{
  /* Return a string of characters from bot to top */
  struct FD_STRING_STREAM s; int i=bot; char buf[128];
  if ((top-bot)*2 > 128)
     fd_raise_exception("Range might be too big");
  else {
    FD_INITIALIZE_FIXED_STRING_STREAM(&s,128,buf);
    while (i < top) {fd_sputc(&s,i); i++;}
    /* We have to copy it because it is in this stack frame. */
    return fd_strdup(s.ptr);}
}

The function fd_printf(fd_string_stream,char *fmt,...) is a version of the printf which takes a string stream as argument and also supports several additional format directives:

The function fd_fprintf takes a FILE * pointer as its first argument. It will also obey any encoding assigned to the file pointer by fd_set_file_encoding. The function fd_xprintf(char *fmt,...) handles exceptional i/o which goes to either the standard error output or a dynamically declared error stream.

The functions fd_notify(char *fmt,...) and fd_warn(char *fmt,...) are both specialized versions of fd_printf which use globally declared handlers to display output to the user. These functions might, for instance, pop up a dialog box rather than doing output to the console. Both functions produce strings which are then passed to handlers. The handler for fd_notify is defined by fd_set_notify_handler: its argument is a function which takes a string and returns void. The handler for fd_warn is defined by fd_set_warn_handler: its argument is also a function which takes a string and returns void. The function fd_disable_notifications() turns fd_notify into a no-op. There is no way to turn off warnings. In the case of both functions, the default action is to do output to the standard error stream (stderr).

Reading from Strings

UTF-8 strings can also be read from using a set of functions which parallel the FILE and XFILE reading functions. In this case, an input string is simply a pointer to a pointer to a UTF-8 string (i.e. fd_utf8 **; as characters are read, the pointer is advanced. The function fd_sgetc(fd_u8char **ss) reads a character from such a stream. The function fd_parse_lisp_from_string(fd_u8char **ss) parses a LISP object from the representation in the string.

Lisp Types

There are four basic kinds of lisp types in FramerD:

The immediate types are:

The primary consed types are:

The primary composite types are:

Foreign types are reference counted C pointers. The pointer of a foreign type can be extracted by the macro FD_CPTR_DATA(x), and an object of foreign type is created by the function fd_make_cptr(type,ptr) where type is one of the legal fd_lisp_type values.

This table lists the foreign types, the kind of pointer which CPTR_DATA, and the chief file where that data structure is defined and used will return:

TypeFD_CPTR_DATA isfile(s)
dtype_server_typestruct FD_SERVER *src/eval/eval.c
pool_typestruct FD_POOL *src/odb/odb.c
index_typestruct FD_INDEX *src/index/index.c
hashtable_typestruct FD_HASHTABLE *src/cons/hash.c
src/fdscript/hashprims.c
hashset_typestruct FD_HASHSET *
input_file_typeFILE *src/fdscript/ioprims.c
output_file_typeFILE *
io_file_typeFILE *
input_string_typestruct FD_STRING_ISTREAM *
output_string_typestruct FD_STRING_STREAM *
mutex_typepthread_mutex_t *src/eval/threads.c
mystery_typestruct FD_MYSTERY *
segmented_stream_typestruct FD_MYSTERY *src/text/text.c
record_stream_typestruct FD_MYSTERY *
tx_closure_typestruct TX_CLOSURE *src/text/match.c

In addition, a number of special `foreign' types are used by the evaluator; this table summarizes them. For the time being, if you have any questions, "Use the Source, Luke".

typeFD_PTR_DATA isDescription
tail_call_type struct FD_VECTOR * Represents a partially evaluated expression, used to allow constant-space tail recursion
cproc_type struct FD_CPROC * a C primitive accessible from FDScript
sproc_type struct FD_SPROC * a Scheme procedure definition, closed in a particular environment
ssproc_type struct FD_SSPROC * a "synchronized" scheme procedure, which is guaranteed to only be running in any one thread at any one time
continuation_type a FD_CPTR to a char * indicates a dynamic return point somewhere up the stack; the continuation passing mechanism uses FramerD's native exception handling to implement its upward continuations
gproc_type struct FD_GPROC * a procedure which returns the empty choice when called recursively on the same arguments; used for certain special sorts of programming
rproc_type struct FD_RPROC * a procedure which uses a remote server to get its value
multiple_value_type struct FD_VECTOR * used to represent multiple values returned from a procedure; note that these multiple values are different from choices
delay_type struct FD_PAIR * a delayed value; the CAR is the delay's thunk, the CDR is the value returned by it and the thunk is freed after itis called.
env_type struct FD_LISPENV * a pointer to an environment structure; when this structure has a ->module field, it is a top level module
quoted_choice_type struct FD_CHOICE * This is used to temporarily suspend non-determinism when passing an argument; a quoted choice is automatically unquoted whenever it is passed to a primitive or bound to a variable

FDScript Evaluator Functions

fd_add_cproc("fdscript-name",arity,c_procedure)
Declares a new primitive in the scripting language with the name "fdscript-name", implemented by c_procedure, and taking arity. Evaluating the expression (fdscript-name arg1 arg2 ...) calls the c_procedure on the result of evaluating each of the arguments.
fd_add_cform("fdscript-name",c_procedure)
Declares a new special form in the scripting language with the name "fdscript-name", implemented by c_procedure. Evaluating the expression (fdscript-name arg1 arg2 ...) calls the c_procedure on the entire expression without evaluating any arguments.
fd_add_nary_cproc("fdscript-name",c_procedure)
Declares a new primitive in the scripting language with the name "fdscript-name", implemented by c_procedure, and taking any number of arguments. Evaluating the expression (fdscript-name arg1 arg2 ...) calls the c_procedure on a list whose elements are the results of evaluating each argi. If the argument evaluations return non-deterministic values, the values are passed as elements of the list, rather than calling c_procedure repeatedly on each combination.
fd_eval(expr)
Evaluates an expression in this process.
fd_eval_loop(input,output)
Enters a read-eval-print loop going to a specified input and output stream.
fd_get_arg(expr,arg_number,default)
Given an expression, returns the arg_numberth element if it exists and the default otherwise. If the default value is FD_VOID, an error is signalled if the argument is not provided.
fd_get_body(expr,offset)
Given an expression, returns the sublist starting after the offsetth element.
fd_interactive_loop(input file stream,output file stream)
Iteratively reads expressions from the input file stream (a FILE *), evaluates them, and outputs them to the output file stream.
fd_load_init(filestring)
Loads and evaluates a file of FDScript expressions.

Writing a Module

To write a C module to link FDScript with some existing library, take a look at the automatic code generator in extras/c-bindings/cbind.fdx. Otherwise, this section offers a few pointers and you can look at samples/c-modules/ for example code.

There are three basic kinds of primitives: cprocs, lexprs, and special forms. A cproc takes some fixed number of arguments, all of which are evaluated; a lexpr takes an arbitrary number of arguments, all of which are evaluted; a special form takes an expression and an environment and does whatever it pleases with it.

Special forms are declared by the function fd_add_special_form(env,"name",cproc) which defines a special form bound to the symbol name in env which will be handled by the C function cproc. The C function cproc should have the signature lisp cproc(lisp expr,fd_lispenv env).

CPROCs are declared by the function fd_add_cproc("name",n_args,cproc) which defines a CPROC that takes n_args arguments (surprise) and is handled by the C function cproc. The C function cproc should have the signature lisp cproc(lisp arg1,lisp arg2,...). The compiler will not warn you if the number of arguments doesn't match. This binds the symbol name to this primitive in the environment env.

LEXPRs come in two flavors and are declared by the function fd_add_lexpr("name",lexpr_type,cprc) where lexpr_type is either NORMAL_LEXPR_ARGS or SPECIAL_LEXPR_ARGS. If the type is NORMAL_LEXPR_ARGS, the evaluator will automatically enumerate any choices passed as arguments (so cproc may be called multiple times). Otherwise, cproc will only be called once, but the argument list it receives may contain non-singular choices. This binds the symbol name to this primitive in the environment env.

A C module normally contains a series of static functions which implement the primitives and an initialization function which calls the function fd_make_module() to make an empty module (which has C type fd_lispenv). It then makes a series of calls to fd_add_cproc and its siblings to populate this environment with siblings. Finally, it calls the function fd_register_module(name,env) on the module to bind name in the global environment to the module where a call to USE-MODULE! can find it. (Alternatively, it can call the function fd_register_restricted_module to bind the module in FDScript's `restricted environment'.)

If the env argument to the fd_add_ functions is NULL, the primitive is defined in the global environment. This will allow for faster accesses, but access cannot be restricted and there is a risk of name collisions.

The Directories of FramerD

The top level of the FramerD directory tree (which you get when you unpack a source tarball or do a CVS checkout) is organized into multiple subdirectories show in this table.

Top Level FramerD directory structure
srccontains the source for the C libraries and executables; the structure of these sources is described below.
include/contains C header files used by the FramerD source files
include/framerd/contains C header files used by external FramerD libraries as well as the FramerD source files
bin/is the directory into which executable files are compiled
lib/is the directory into which static and shared libraries are compiled
etc/ contains the Emacs Lisp (ELISP) interface to FramerD, the FDScript installation scripts, and various shell scripts for server management.
extras/ contains subdirectories for various FDScript extension libraries written in C. Each has its own makefile and produces static and dynamic libraries
extras/c-bindings The file cbind.fdx implements a package which automatically generates interfaces from FDScript to C libraries. The individual `.fdx' files in this directory implement various bindings for common libraries.
docs/ contains documentation for FramerD in HTML, together with a handful of .png format images.
tests/contains a FramerD test suites, including Aubrey Jaffer's R4RS test suite, custom tests for FramerD itself, and regression tests to catch the return of past bugs.
tests/gabrielcontains Scheme versions of the Gabriel benchmarks for evaluating FDScript performance
scripts/ contains FDScript programs to be installed as command line utilities. These utilities include pool and index maintenance utilities as well as command line tools for browsing FramerD databases.
samples/contains subdirectories of various collections of sample code for servers, shell commands, web scripting, and email parsing

In addition, the top level directory contains configuration and make files (configure, configure.in, and makefile.in) as well as text files describing how to compile FramerD (BUILD), how to install it (INSTALL), news about changes (README), and a copy of the GNU General Public License (LICENSE).

Conceptually, the FramerD sources can be divided into six modules, summarized in this table.

Structure of FramerD source modules
directorieslibrariesDescriptionheader files
src/oslibdtypes a system module provides for OS-independent internationalization, environment access, and file system operations include/framerd/os.h
include/framerd/except.h
include/framerd/strstream.h
src/cons a data module provides the basic LISP data structures, printed and binary object i/o, and some derived structures, including hashtables and slotmaps include/framerd/cons.h
include/framerd/lisp.h
src/odb
src/index
src/eval
libframerd the FramerD core provides management for object (OID) databases, persistent associative indices, and a minimal evaluator for a subset of Scheme include/framerd/odb.h
include/framerd/index.h
include/framerd/eval.h
src/schemelibfdscript the Scheme module provides most of the R5RS scheme functions, including rational, complex, and arbitrary precision arithmetic include/framerd/bignum.h
include/framerd/bignmint.h
src/fdscript the FDScript module provides further advancec functions, including operating system access, bindings for internal utilities in the system, data and FramerD modules, formatted output, generic sequences, and console interaction  
src/textlibfdtext the text module provides pattern matching and parsing functions for natural and artifical languages, especially intended for implementing transducers between existing programs and databases and FramerD. include/framerd/fdtext.h
src/exe
src/misc
  the executables themsleves which are linked against the libraries above; the directory src/misc contains code used with the server and CGI executables. A description of the standard executables can be found here  

The "standard executables" in FramerD are compiled from files in src/exe, linked against the above libraries. The server and web gateway executables are also linked together with the files src/misc/server.c and src/misc/cgiparse.c. These standard executables are summarized in this table.

Standard FramerD executables
print-dtype prints and ASCII representation of a DTYPE stored in a file
make-dtype makes a DTYPE in a file given an ASCII representation from the command line
dtcall does a remote procedure call using the DType protocol.
pool-get prints the value of an OID in a pool
index-get prints the values associated with a key in an index
analyze-index analyzes and index file for hashing properties and statistics
fdscript runs an FDScript interpreter, either interactively or on a command file; this is used to implement command-line access to FramerD databases
fdcgi runs an FDScript interpreter, intended to use with a web server's common gateway interface to provide WWW-hosted interaction with FramerD databases
fdserver starts a FramerD server on a specified port, which may provide access to pools or indices in local files

The Files of FramerD

src/cons/except.c implements basic exception handling based on the C runtime setjmp and longjmp
src/cons/data.c implements data types such as lists and integers and the facilities for implementing new datatypes in C
src/cons/hash.c interned datatypes (symbols and object identifiers (OIDs)) which rely on them
src/cons/xdata.c datatypes, including vectors, slotmaps, bignums, unicode strings, etc
src/cons/io.c
src/cons/dtio.c
src/cons/txio.c
implements the binary DTYPE format for transmitting data across the network and storing it persistently in disk files; it is included twice in the file io.c to implement I/O to and form buffers in memory and I/O using the C stdio library.
  • src/cons/io.c provides a wrapper for the binary DTYPE functions as well as functions for producing and parsing printed representations of objects. It also includes the functions for getting miscellaneous enviromental information (including accessing the WIN32 registry)
  • src/os/network.c implements basic generic network access functions across operating systems
    src/os/fdprintf.c with a printf syntax extended for the presentation of lisp objects. Output can be either directed to files or to the string streams used throughout FramerD. This file also implements exceptional i/o facilities for managing reports of unexpected events.
    src/os/i18n.c provides an OS independent layer for dealing with Unicode strings and character set conversions
    src/os/envfns.c provides an OS independent layer for dealing with environment queries, including time and system functions
    src/os/filefns.c provides an OS independent layer for dealing with file access and information
    src/odb/odb.c implements the object database which maps OIDs to their values, whether stored in local files or remote servers; it also manages the locking and unlocking of OIDs and the organization of OIDs into disjoint pools.
    src/odb/pools.c implements the management of the pools into which the object address space is divided.
    src/odb/file-pool.c implements access to subsets of the object database stored in disk files, called file pools
    src/odb/network-pool.c implements access to subsets of the object database stored on remote servers, called network pools
    src/odb/frames.c implements frames which are OIDs whose values are slotmaps implementing an attribute value structure with the possibility for complex inferences over some of their values.
    src/index/index.c implements access to persistent indices mapping Scheme objects to each other and especially matching key objects to OIDs
    src/index/file-index.c implements access to on-disk hash tables as indices.
    src/index/network-index.c implements access remote DTYPE servers used as indices.
    src/index/search.c combines the frames implemented in the object database with the indexing facility to do automatic or semi-automatic frame indexing
    src/eval/eval.c contains the evaluator kernel and a small set of evaluator related functions.
    src/eval/ndeval.c contains handlers for dealing with non-determinism in evaluation and application
    src/eval/mini.c contains basic core functions (no binding constructs) to be used by FramerD inference methods
    src/eval/lambda.c implements environment structures and lambdas to support user definitions.
    src/eval/sandbox.c implements basic symbol level access in order to support limited-capability interpreters for more secure FDScript environments
    src/eval/threads.c contains support for multi-threaded programming, including mutexes and spawning new threads.
    src/eval/load.c implements procedures for loading disk files containing FDScript code.
    src/scheme/arith.c contains arithmetic primitives
    src/scheme/bignum.c contains the code for arbitrary precision integers, inherited from MIT Scheme
    src/scheme/strings.c contains special string manipulation primitives
    src/scheme/pairs.c contains special pair/list manipulation primitives
    src/scheme/characters.c contains special character access and manipulation primitives
    src/scheme/reflect.c contains forms for environment manipulation
    src/scheme/special.c contains Scheme and Common Lisp special forms
    src/scheme/cxr.c contains list access primitives of the form "C[AD]+R"
    src/scheme/ioprims.c contains primitives for binary and character I/0, including the R4RS I/0 primitives
    src/fdscript/prims.c contains a miscellaneous primitives for FDScript
    src/fdscript/printout.c implements the PRINTOUT facility for producing formatted output.
    src/fdscript/seq.c contains special sequence functions inherited from Common Lisp
    src/fdscript/osprims.c contains primitives for operating system access
    src/fdscript/framerd.c contains primitives for accessing FramerD objects, frames, pools, and indices
    src/fdscript/maint.c contains primitives for maintaining FramerD pools and indices
    src/fdscript/hashprims.c contains primitives for using hashtables and hashsets