Memory management

First I would like to let you know there has been a new release of the compiler for Hyper, version 0.3.36. I did not announce that version on my blog.

Now, about memory management. As you might already now, the language will use a garbage collector. As a consequence of that, classes cannot have a destructor because it cannot be guaranteed that such a destructor would really be executed when an object instance's memory is reclaimed. When a short running program exits, the garbage collector might even not have been run. This way, the garbage collector actually serves to pretend we have an infinity amount of memory. But people don't always like it that way, they want deterministic construction/destruction for some things like resources managed by the operating system. And I agree, though Hyper is not a language for doing low level OS stuff it can be necessary to have deterministic resource management for some cases. So I will introduce a way to do that.

First we need a bit of background. There is a feature I thought I have already explained on this blog, but I can't find it anywhere. So I will describe it briefly, and write a full explanation later. This feature is called "restricted pointers", and a shorter name might be "references". Such a reference is like a pointer, except that it cannot be stored outside of the stack and the guarantee that it will never be null. References can be used to point to a temporary object on the stack. A pointer can be implicitly converted to a reference but not the other way around. Temporaries can no longer be converted to pointers but only to references. This implies that when you see a pointer it has to point to an object on the free store. References are not yet implemented in the compiler, but they will have to be in order to get headache free memory management; they prevent the use of a pointer that points to a no longer existing temporary.

I already wrote something about RAII some time ago, but I realized that it would not be good enough for safe deterministic destruction, because the user can use the object without "scope". My new solution is called "auto classes". Such a class is declared with the "auto" keyword, and it must have a destructor. The syntax for the destructor could be "procedure auto()" or "procedure ~new()", I'm not sure yet. Any instance of an "auto class" can only be stored on the stack, or in a (non-static) field of another "auto class". It cannot be created on the free store, as it would not be possible to provide a guarantee for the execution of the destructor then. An object of such a class would need to count internally how many copies of it are still live in order to decide when its resource can be released. My plan is to provide an abstract base class that implements this behaviour so that users don't have to write a reference counter for every auto class they need; they would just inherit from the base class and implement a procedure that specifies what to do when the last instance dies.

A slightly different approach would be a resource-managing class that cannot be changed. So I propose the "auto const class", an auto class that is always const, it does not have an (accessible) assignment operator and all its procedure must be const as well. Its copy constructor can be made private in order to have a RAII class that does not need a reference count because it is the only reference.

Aside from memory management I also propose a (non-auto) "const class". This type of class is always const (thus immutable) and is in the style "create and never modify", like the Java "String" class. It can only be constructed on the free store.

An extension to the type system

Why change Hyper's type system? The main reason is arrays, or more specifically the array index operator. When you have an array of 'int', the return type of the index operator needs to be a pointer to an 'int' in order for you to be able to change the numeric value in the array. (Currently this is not the case yet; the array index operator returns a plain 'int' for such an array. That will be fixed of course.) But what when you have an array of pointers to 'int'? What will be the index operator's return type then? Logically it has to be something like pointer to pointer to int, because of the extra level of indirection that is needed. After all, you should be able to change the int pointer. But at this time the language disallows multiple levels of pointers!

My first idea to solve this problem was to introduce "inout return types". It would provide an extra (hidden) reference to the actual type, as it is done for inout parameters. But I decided not to do that, as it would be a quirky solution. And then someone pointed out that the current pointer system is weird and makes source code unreadable because of the implicit (de)referencing done by the compiler. So I thought of changing the type system to be more like C++, with explicit referencing/dereferencing for most things. Unfortunately it seemed to me that the changes would lead me to something that was almost identical to C++ but a bit more complicated. I decided that this wasn't an option either.

I looked back to the actual problem and my first idea of 'inout return types'. What I have come up with is a bit similar to that solution, only less quirky. The main idea is introducing a second pointer level, thus allowing a pointer-to-pointer-to-class type to be declared in some places. This introduces an ambiguity, namely: to what pointer level does the pointer assignment etc. apply to? Well, the second (i.e. top-level) pointer is only used as a reference to the single pointer; you need to be able to change the single pointer as in the array-of-pointer-to-int example. That means all pointer operations would need to work on the single pointer, the one that points to the 'int' in the example. The double pointer would serve like a reference in C++, only initialized once and always dereferenced when used in an expression. The usage of such a double pointer is not universal; it would not be allowed for parameter types, because input parameters don't need a second level of indirection and because inout parameters already provide an implicit second pointer for a pointer parameter. A double pointer can be useful for return types, for variable types, and maybe for fields as well.

Some examples now:

var i : int = 22
var p: *int = i  # points to i
var pp: **int = p  # points to p
var b : bool
b = (p = 22)  # yes, p equals 22
b = (pp = 22)  # yes, pp equals 22
b = (pp $= p)  # yes, pp equals p
p = 47  # change i to 47
pp = 84  # change i to 84
p $= new int(55)  # p no longer points to i
pp $= new int(31)  # p changed again, points to 31

As you see in the example pp can not be changed anymore; it is a reference to p and all pointer assignments done on pp will be therefore applied to p.

This new functionality allows us to write a class that serves as a pointer-to-int:

class IntPtrArray
public:
var fP : [10] * int  # array has a fixed size of 10 ints

const procedure size() : nat
  return 10
end

operator[](index:nat) : **int
  # return a reference to the int pointer
  return fP[index]
end

const operator[](index:nat) : *int
  # no double pointer return needed (array is const)
  return fP[index]
end
end

I'm not sure when I will implement this feature, but unless I get some serious objections against this the feature will be added to the language.

website moved

My website has moved recently. The new URL for Hyper is: http://users.edpnet.be/hyperquantum/hyper/

The project is somewhat stalled right now. I don't have much time these days, and the language's type system needs to be reworked a bit.

Hyper compiler 0.3.35 released

It has been a long time, but here's another release. This one is nothing spectacular; most changes and improvements are internal and not visible to a user. I started to use the Boost libraries. The compiler and highlighter programs now use Boost's program_options library for parsing command-line parameters.

The lack of interesting new features in the front end is because I have worked on the back end as well. This work is in a very premature state, so you won't see a full compiler anytime soon.