"Final" thoughts on currently outstanding language changes:
In summary, I am looking at replacing the preprocessing (#) stuff with a package-system, C#-ish user-defined attributes with controlled access/manipulation of the syntax tree, compiler variables, etc. Some of these changes are large (surprise!), but I am presenting them
after doing a lot of thinking about how to do things and what is necessary. Also, a few smaller changes / loose ends I've tied down.
Package System:
A package consists of (multiple) source files (.alp) and one package (.apg) file, all of which must start with the same package declaration ("package Foo;"). Packages are "compiled" into a single unit containing meta-data about the code (compiled syntax tree, and assembly code to whatever extent possible). The package file lists other files (or directories) that are included in the package, and may contain declarations that apply to the entire package.
"Importing" a package within a file gives a that file access to the code in that package. Importing within a package file does so for all files in the package. The compiler only loads (compiles, or reads the already compiled data for) a package once no matter how many times it is "imported". The code of an imported package remains in the "namespace" of the package (e.g. "import Foo; ... new Foo.Bar()), but specific contents of a package can also be imported (e.g. "import Foo.Bar; ... new Bar()).
Packages (and their contents) must reside in predictable locations (nearby directories, or in a (configurable?) default directory that the compiler always checks, or in some directory specified by a package as a place to look for that package's imports). If a file is part of a package, then attempting to compile it will cause the compiler to look for the package file (nearby) and compile the package.
"Inner" packages (packages "within" packages) depend on (automatically "import") the parent package, and must be referenced inside the package-file of the parent package. For example, package "Foo.Bar" refers to package "Bar" inside of package "Foo". "Foo" can be imported without "Bar" ever being imported, but importing "Foo.Bar" also imports "Foo". Additionally, if "Foo" is imported, then "Bar" will automatically be imported if any reference to "Foo.Bar" is made (this is why the parent package must be "aware" of inner packages).
Packages are first-class objects. That is, all functions are "methods" of the package they reside in (if not in a class or other object), packages may inherit from classes and compose ("mix in") traits to bring in code (brings code directly into the "namespace" of the package), may contain constructors for the whole package, and even be passed around as objects. (Packages can not be inherited FROM, though; they are not classes). This works without extra overhead, because the IX register will be reserved just for the "this" of methods, and the "methods" of a package (or of any singleton object) can ignore the "this" parameter because the "members" can already be accessed directly (because they are "static", or only exist in one place).
Attributes:
An attribute is like a compile-time class, which can be used to attach meta-data to elements of code (variables, classes, objects, traits, functions, packages, etc.). This meta-data is exposed in the syntax-trees of compiled code, and thus can be accessed by anything that can interface with the compiler or with compiled packages.
Attributes may also contain compile-time ("interpreted only") initialization code that interface directly with the compiler (e.g. get/set compiler properties) and the code that it is attached to (e.g. prepend other code). This init code will be normal code as found anywhere else in the program, but will take a "this" parameter which maps to the syntax-tree item that it is attached to.
For example, different attributes may be used to flag data to use for a program "icon" and functions to handle "update" and "render" parts of a game kernel; and then another attribute (or a package constructor) may contain initialization code to search for items with these attributes and, if found, insert a sort of "program header" containing these items (by inserting the "code" for it at the start of the main function). Or perhaps attributes can be made to implement pre-/post-conditions.
I'd have to decide what operations to allow (inserting code before/after functions, other modifications to the syntax tree, etc.), and provide some "Compiler" object to interface with the compiler (get properties given to the compiler, issue compile-time errors, etc.). Attributes themselves would have built-in methods to query for which items they are used on, etc. I may also want to add methods for basic file I/O, prompts, etc. (for example, read a program "icon" from a file, or interface with an external program which could be included as a resource file in a package to begin with).
After parsing all the source code and builds a syntax tree, but before "compiling" it all, the compiler will then go through and initialize all the attributes on things, which in turn may make other modifications to the syntax tree. (This is why "compiled" packages might not always be able to contain the "fully" compiled assembly code for everything, because it may depend on how it is used when it gets imported).
Code: // Attribute declarations:
attr ProgramIcon; // Has no properties on it
attr MyAttrib(myProp) { ... } // has one property and init code
[ProgramIcon] // Attaching a "ProgramIcon" to myIcon
bool[10,10] myIcon = { { ... }, { ... } , ... };
[MyAttrib(true)] // Attach a "MyAttrib" to someFunc,
func someFunc() { ... } // will init with myProp = true;
[MyAttrib(myProp=true)] // Again, but with named argument
func someOtherFunc() { ... }
Package-level "if":
The "#if" preprocessor-directive would be replaced with a regular "if" statement which can appear at the package level, with the condition using normal code (but most likely referring to things related to attributes or compiler properties as exposed through some compile-time-only "Compiler object"), with semantics that the containing code is only to be included if the condition is true (or removed/ignored if false). A similar "if" inside a function already carries the same semantics:
Code: package Blah; // Just to illustrate package-level context
if(Compiler.Device == TI83PLUS) {
func FooFor83P() { ... }
class ClassFor83P { ... }
}
else {
func FooOtherwise() { ... }
...
}
func Something() {
if(Compiler.Device ... ) // this would normally be
... // evaluated at compile-time anyway (same semantics)
}
This feature would allow items to be removed/added from the syntax tree dynamically without relying on a preprocessor (another built in "language"), and without requiring me to provide "Compiler methods" to make such modifications otherwise (because then I'd have to restrict when/where that code could be used).
Note: I still have to decide whether these get evaluated before or after attribute initializations, or if they can "cycle" (and how).
EDIT: There are two kinds of "package-level" ifs: those that can be handled in one static pass (as above), and those that depend on type-parameters and must be handled per usage (below):
Code: class Foo<T> { // Foo is parameterized on some type "T"
int presentInFooForAllTs; // Foo has this for any T
if(something about T) {
int OnlyForFomeTs; // Foo only has this when stated
}
if(something Not About T) {
int HereOrNotButSameForAll // Whether Foo has this depends on the condition, but is the same for any T
}
}
Situations like this^ (where members of something depend on usage) cause inconsistencies which make the "wild-card" (Foo<?>) behavior (described below) difficult or impossible. I will have to decide whether to disallow such inconsistencies to exist (e.g. every "branch" of those "ifs" must result in something equivalent), or find some other way to limit/modify the "wild-card" behavior to work with/around/within such inconsistencies.
Classes support single-inheritance:
Code: class Base {
func Blah() { ... } // non-virtual, called directly
vfunc Foo() { ... } // virtual (stored in vtable)
vfunc Empty(); // "pure virtual" / "abstract"
}
class Sub : Base { // Sub inherits from Base
vfunc Foo() { ... } // Override Base definition of Foo
vfunc Empty() { ... } // Provide implementation for Empty
val Bar b { vfunc Inner(..){..} } // val members (stored by value) can have overrides at declaration, with direct access to the containing class (Inner has direct access to all of Sub).
}
...
Base b = someSub; b.Foo(); // calls Sub.Foo
Other small changes:
Code: // Overloading the [] operator (presence of return-type):
func [](int a,..,z,val) { ... } // (set) this[a,..,z] = val;
func [](int a,..,z):int { ... } // (get) this[a,..,z];
asm { // The new "asm" command (no #preprocessor):
ld a,$(someVarFromCode); // instructions end with ;
call foo; // same syntax for comments as elsewhere
}
// Enums can be "of" any type with values of that type:
enum IntValues { A = -1, B = 2, C = 300 } // infer type as int
enum<Foo> FooValues { A(1), B(2), C(3) } // or A = Foo(3), etc.
// Equality operators == and "is":
A == B; // Compare primitive values or references (overridable)
A is B; // Test if A is an instance of B (not overridable)
trait T { func ==(...); } // allowed in traits
Code: // Singleton objects:
object Foo { // Like "class", but "Foo" is also the only instance
func new(...) { ... } // Objects and Packages can have constructors (only called if the entity is ever used).
}
class Foo { ... } // "object Foo" holds the "static" members of Foo, but is also an object in of itself (can inherit, etc.) Foo can implement traits by virtue of functions in "object Foo"
// Type-parameters: wild-cards, contravariance/covariance:
List<Foo> foos; // Compiler makes a version of "List" code for "Foo"s
List<?> things; // Version of "List" with extra references to code
things = foos; // Now things holds a reference to foos AND the Foo code
List<in Foo> // Can reference a list of any one type that Foo inherits from
List<out Foo> // Can reference list of any one type that inherits from Foo
Less important features that I may consider later on:
- Ability to test exec-point in yieldy funcs (something akin to "MoveNext" and "Current" in C# enumerators)
- A "default value" implementation (default(Foo) is a "Foo" will all "zero" values / nulls; for use with templated/generic types)
- Explicit non-null types (NullableType, NonNullable!), and nullable equivalent (Blah, NullableBlah?)
- "Trust me that it's not-null" operator: Foo!! (error if compiler can prove you wrong)
- Var-args implementation? (use the stack, or implement as array?)
- Class literals as a short-hand for calling an "Add" or "Append" method auotmatically:
new Dict<string,int> { { "Hello", 5 }, { "Hi", 6 } }; // Dict.Add(string,int)
- Literals for other things (regex, floating-point), each require a class implementation to exist for it