Fork me on GitHub

March 15, 2015

0.5.1: New console and errors support

This new release brings many new features, improvements and some bugfixes that will make Red more usable, especially for newcomers. The initial intent for this release was just to replace the existing console implementation, but it looked like the right time to finally implement also proper general error handling support.

New console engine

The old console code we were using so far for the Red REPL was never meant to last that long, but as usual in software development, temporary solutions tend to become more permanent than planned. Though, the old console code really needed a replacement, mainly for:
  • removing the dependency to libreadline and libhistory, they were creating too many issues on the different Unix platforms, so became troublesome for many newcomers.
  • having a finer-grained control over keystrokes on text input, in order to implement convenient features like word completion.
  • having a bigger platform-independent part, so that we can add any kind of backends, like GUI ones, without duplicating too much code.
So, the new console code gets rid of third-party libraries and runs only on what the OS provides. The new features are:
  • built-in history, accessible from system/console/history
  • customizable prompt from system/console/prompt
  • word and object path completion using TAB key
  • ESC key support for interrupting a multi-line input
Other notable console-related improvements:
  • about function now returns also the build timestamp.
  • what function has now a more readable output.
  • Console output speed on Windows is now very fast, thanks to the patch provided by Oldes for buffered output.
The console code is not in its final form yet, it needs to be even more modular and wrapped in a port! abstraction in the future.

Errors support

Red now supports first class errors as the error! datatype. They can be user-created or produced by the system. The error definitions are stored in the system/catalog/errors object.
 red>> help system/catalog/errors
 `system/catalog/errors` is an object! of value:
     throw            object!   [code type break return throw continue]
     note             object!   [code type no-load]
     syntax           object!   [code type invalid missing no-header no-rs-h...
     script           object!   [code type no-value need-value not-defined n...
     math             object!   [code type zero-divide overflow positive]
     access           object!   [code type]
     user             object!   [code type message]
     internal         object!   [code type bad-path not-here no-memory stack...

User errors can be created using make action followed by an error integer code or a block containing the category and error name:
 red>> make error! 402
 *** Math error: attempt to divide by zero
 *** Where: ???

 red>> make error! [math zero-divide]
 *** Math error: attempt to divide by zero
 *** Where: ???

These examples are displaying an error message because the error value is the returned value, we still need to implement a full exception handling mechanism using throw/catch natives in order to enable raising user errors that can interrupt the code flow. The error throwing sub-system is implemented and used by the Red runtime and interpreter, just not exposed to the user yet.

Errors can be trapped using the try native. An error! value will be returned if an error was generated and can be tested using the error? function.
 red>> a: 0 if error? err: try [1 / a][print "divide by zero"]
 divide by zero
 red>> probe err
 make error! [
    code: none
    type: 'math
    id: 'zero-divide
    arg1: none
    arg2: none
    arg3: none
    near: none
    where: '/
    stack: 3121680
 ]
 *** Math error: attempt to divide by zero
 *** Where: /
Currently the console will display errors if they are the last value. That behavior will be improved once the exception system for Red will be in place.

Errors when displayed from compiled programs, provide calling stack information to make it easier to locate the source code where the error originated from. For example:
    Red []
    
    print mold 3 / 0
will produce the following error once compiled and run:
    *** Math error: attempt to divide by zero
    *** Where: /
    *** Stack: print mold /

SORT action

Sorting data is now supported in Red, in a polymorphic way as in Rebol. The sort action is very versatile and useful. Let's start from a basic example:
    scores: [2 3 1 9 4 8]
    sort scores
    == [1 2 3 4 8 9]
As you can see, sort modifies the argument series, you can keep the series unchanged by using copy when passing it as argument:
    str: "CgBbefacdA"
    sort copy str
    == "aABbCcdefg"
    sort/case copy str
    == "ABCabcdefg"
    str
    == "CgBbefacdA"
By default, sorting is not sensitive to character cases, but you can make it sensitive with the /case refinement.

You can use /skip refinement to specify how many elements to ignore, it's handy when you need to sort records of a fixed size.
    name-ages: [
        "Larry" 45
        "Curly" 50
        "Mo" 42
    ]
    sort/skip name-ages 2
    == ["Curly" 50 "Larry" 45 "Mo" 42]
The /compare refinement can be used to specify how to perform the comparison. (It does not yet support block! as argument)
    names: [
        "Larry"
        "Curly"
        "Mo"
    ]
    sort/compare names func [a b] [a > b]
    == ["Mo" "Larry" "Curly"]
Combining it with /skip refinement, you can do some complex sorting task.
    name-ages: [
        "Larry" 45
        "Curly" 50
        "Mo" 42
    ]
    sort/skip/compare copy name-ages 2 2    ;-- sort by 2nd column
    == ["Mo" 42 "Larry" 45 "Curly" 50]
The /all refinement will force the entire record to be passed to the compare function. This is useful if you need to compare one or more fields of a record while also doing a skip operation. In the following example, sorting is done by the second column, in descending order:
    sort/skip/compare/all name-ages 2 func [a b][a/2 > b/2]
    == ["Curly" 50 "Larry" 45 "Mo" 42]
Sort uses Quicksort as its default sorting algorithm. Quicksort is very fast, but it is an unstable sorting  algorithm. If you need stable sorting, just add /stable refinement, it will then use Merge algorithm instead to perform the sort.

New datatypes

A couple of new datatypes were added in this release, mostly because of internal needs in Red runtime to support the new features.

The typeset! datatype has been fully implemented, and is on par with the Rebol3 version. A typeset! value is a set of datatypes stored in a compact array of bits (up to 96 bits). Datatype lookups are very fast in typesets and they are mostly used internally for runtime type-checking support. The following actions are supported on typeset! values: make, form, mold, and, or, xor, complement, clear, find, insert, append, length?. Comparison operators are also supported.

A preliminary implementation of the vector! datatype is also part of this release. A vector! value is a series of number values of same datatype. The internal implementation uses a more compact memory storage format than a block! would do, while, on the surface, behaving the same way as other series. Only 32-bit integer values can be stored for now in vectors. The following actions are supported by vector! values: make, form, mold, at, back, head, head?, index?, insert, append, length?, next, pick, skip, tail, tail?. The implementation will be completed in future releases.

Runtime type checking support

It has finally being implemented, as proper error handling support is now available. So from this release on, function arguments types will be check against the function specification and non-conforming cases will result in an error. Return value type-checking will be added later.

The type-checking might break some existing Red code around that was letting silently pass invalid arguments, so check your code with this new release before upgrading.

The compiler does not do any type checking yet, that will be added at a later stage (though, don't expect too much from it, unless you annotate with types every function exhaustively).

Also notice that the runtime type-checking implementation is making the Red interpreter a little bit faster, thanks to a new optimized way to handle function specification blocks (an optimized spec block is cached after first call, resulting in much faster processing time afterwards).

Red/System improvements

Exceptions handling has been improved, introducing the catch statement allowing to catch exceptions using an integer filtering value. Here is a simple example in the global context:

    Red/System []

    catch 100 [
        print "hello"
        throw 10
        print "<hidden>"
    ]
    print " world"
will output
    hello world

The integer argument for catch intercepts only exceptions with a lower value, providing a simple, but efficient filtering system.

In addition to that, uncaught exceptions are now properly reporting a runtime error instead of passing silently. This new enhanced low-level exception system is supporting the new higher-level Red error handling system.

A couple of new compiler directives have been also added in order to strengthen the interfacing with Red layer:
    #get <path>
The #get directive returns a red-value! pointer on a value referred by a Red object path. This is used internally in the runtime to conveniently access the Red system object content from Red/System code. This directive will be extended in the future to access also words from Red global context.

    #in <path> <word>
The #in directive returns a red-word! pointer to a Red word bound to the object context referred by path.

What's next?

In addition to many minor pending improvements, we will be working on a minor release that will introduce the Redbin format for accurately serialize Red values in binary form. Redbin format will be used to make the compilation process much faster, as it currently slows down pretty quickly as the Red-level environment code size grows up.

Enjoy this new release! :-)

January 15, 2015

Dream big, work hard and make it happen!

Today is a big day for the Red team and all the Red followers.

After four years of hard work on building our dream tool, I announce today the creation of a new company, Fullstack Technologies. The company has raised $500,000 from InnovationWorks and GeekFounders, Chinese VC early-stage investors. This money will help us fuel the launch of Red this year, and spread it everywhere, especially in the mobile market. The mission of Fullstack Technologies is to provide to individual developers and corporations, a simpler and much more productive software creation solution, reducing drastically both costs and development time.

The company has its headquarters in Zhongguancun, Beijing (Chinese's "Silicon Valley"). I am CEO now (once more) and Xie Qingtian (qtxie) is joining the company as tech lead. We plan to recruit more people in the next months.

I want to use this opportunity to express my deepest gratitude to all the people in the community who have helped me, contributed to Red and supported my work with donations since the beginning. I simply could not have made it so far without you.

Also, even if the funding needs are now covered, the Red community still has a major role to play in helping us build the best possible tool and spreading it. Fullstack Technologies intends to work closely with the community, by providing some contracted jobs, bounties and sponsoring for helping spread Red locally.

People who follow closely my work on Red know that I have a strong vision and big ambitions for it, now I have the means to make all that happen! This journey is getting even more exciting. :-)



December 24, 2014

Objects implementation notes

I would like to share some notes about how some of the object features were implemented in the Red compiler. As these are probably the most complex parts in the Red toolchain right now, I thought that it would be worth documenting them a bit, as it can be useful to current and future code contributors.

Shadow objects and functions

Reminder: the Red toolchain is currently written entirely in Rebol.

During the work on object support in the Red compiler, I realized that I could leverage the proximity of Red with Rebol much deeper than before, in order to more easily map some Red constructs directly to Rebol ones. That's how I came up with the "shadow" objects concept (later extended to functions too).

It is pretty simple in fact, each time a Red object is processed from the source code, an equivalent, minimized object is created by the compiler in memory and connected to a tree of existing objects in order to match the definitional scoping used in the Red code.

Here is an example Red source code with two nested objects:
    Red []
    
    a: object [
        n: 123
        b: object [
            inc: func [value][value + n]
        ]
    ]
Once processed by the compiler, the following shadow objects are created in memory:
    objects: [
        a object [
            n: integer!
            b: object [
                inc: function!
            ]
        ]
    ]
But it does not just stop there, the body of the Red object is bound (using Rebol's bind native) to the internal Rebol object, in such way that the definitional scoping order is preserved. So the Red code is directly linked to the Rebol shadow objects in memory. The same procedure (including the Red code binding to Rebol objects part) is applied to all compiled functions, which context is represented as a nested Rebol object in the compiler's memory.

If you get where I am heading, yes, that means that resolving the context of any of the words contained in a Red object/function body becomes as simple as calling Rebol's bind? native on the word value. (Remember that Red source code is converted to a tree of blocks before being compiled). The bind? native will return one of the Rebol's objects, that can then be used as a key in an hashtable to retrieve all the associated metadata.

I wish I had come up with that simple method when I was implementing namespaces support for Red/System. I think that I will rework that part in Red/System in the future, reusing the same approach in order to reduce compilation times (namespaces compilation overhead is significant in Red/System, roughly taking 20% of the compilation time).

Choosing Rebol as the bootstrapping language for Red, shows here its unique advantages.


Dynamic invocation

Processing path values is really difficult in Red (as it would be in Rebol if it had a compiler). The main issue can be visualized easily in this simple example:
    foo: func [o][o/a 123]
Now if you put yourself in the shoes of the compiler, what code would you generate for o/a ?... Could be a block access, could be a function call with /a as refinement, could be an object path accessing a field, could be an object path calling the function a defined in the object. All these cases would require a different code output, and the compiler has no way to accurately guess which one it is in the general case. Moreover, foo can be called with different datatypes as argument, and the compiled code still need to account for that...

One method could be to generate different code paths for each case listed above. As you can guess, this would become quickly very expensive to manage for expressions with multiple paths, as the possible combinations would make the number of cases explode quickly.

Another, very simple solution, would be to defer that code evaluation to the interpreter, but as you cannot know where the expression ends, the whole function (or at least the root level of the function) would need to be passed to the interpreter. Not a satisfying solution performance-wise.

The solution currently implemented in Red compiler for such cases, is a form of "dynamic invocation". If you go through all cases, actually they can be sorted in two categories only:

a) access to a passive value
b) function invocation

Only at runtime you can know which category the o/a path belongs to (even worse, category can change at each foo function call!). The issue is that the compiler generates code that evaluates Red expressions as stack manipulations (not the native stack, but a high-level Red stack), so the compiler needs to know which category it is, so it can:
  • create the right corresponding stack frames.
  • consume the right number of arguments in case of a function invocation.
Basically, the generated Red/System code for the foo function would be (omitting prolog/epilog parts for clarity):

For a) case:
    stack/mark-native ~eval-path 
    stack/push ~o
    word/push ~a 
    actions/eval-path* false 
    stack/unwind
    integer/push 123
    stack/reset
For b) case (with /a being a refinement):
    stack/mark-func ~o
    integer/push 123
    logic/push true               ;-- refinement /a set to TRUE
    f_o
    stack/unwind
As you can see, the moment where the integer value 123 is pushed on stack for processing is very different in both cases. In case a), it is outside of the o/a stack frame, in case b), it is part of it. So what should the compiler do then...looks unsolvable?

Actually some stack tricks can help solve it. This is how the compiler handles it now:
  • The stack can either overwrite new expressions (default) or accumulate them.
  • At each level of a path evaluation, a check for a function result is applied. When a function is detected, it is pushed on stack and a new stack frame is opened to gather the required arguments. Such function is named a "dynamic call" in this context.
  • Some stack primitives (like stack/reset) are modified to not only support the overwritting/accumulative modes, but also check if the arity for the pending dynamic call has been fulfilled, and when appropriate, run the deferred function call, clean-up the stack and revert to the default overwritting mode.
This is the code currently produced by the Red compiler for o/a 123:
    stack/push ~o
    either stack/func? [stack/push-call path388 0 0 null] [
        stack/mark-native ~eval-path
        stack/push stack/arguments - 1      ;-- pushes ~o
        word/push ~a
        actions/eval-path* false
        stack/unwind-part
        either stack/func? [
            stack/push-call path388 1 0 null
        ][
            stack/adjust
        ]
    ]
    integer/push 123
    stack/reset
This generated code, with the help of the dual-mode stack, can support evaluation of o/a whatever value the path refers to (passive or function). stack/func? here checks if the stack top entry is a function or not. There is a little performance impact, but it is not significant, especially in respect to the high flexibility it brings.

So far so good. What happens now if the path is used as argument of a function call:
    foo: func [o][probe o/a 123]
The outer stack frame that probe will create then becomes problematic, because it will close just after o/a, preventing it to fetch eventual arguments when o/a is a function call...so back to the drawing board? Fortunately not, we can apply the same transformation for the wrapping call and defer it until its arguments have been fully evaluated. This is the resulting code:
    f_~path389: func [/local pos] [
        pos: stack/arguments 
        stack/mark-func ~probe 
        stack/push pos 
        f_probe 
        stack/unwind
    ] 

    stack/defer-call ~probe as-integer :f_~path389 1 null
    
    stack/push ~o
    either stack/func? [stack/push-call path388 0 0 null] [
        stack/mark-native ~eval-path
        stack/push stack/arguments - 1
        word/push ~a
        actions/eval-path* false
        stack/unwind-part
        either stack/func? [
            stack/push-call path388 1 0 null
        ][
            stack/adjust
        ]
    ]
    ------------| "probe o/a"
    integer/push 123
    stack/reset
As you can see, it gets more hairy, but still manageable. The outer stack frame is externalized (into another Red/System function), so it can be called later, once the nested expressions are evaluated.

That said, dynamic calls still need a bit more work in order to support routine! calls and refinements for wrapping calls. Those features will be added in the next releases. Also, the gain in flexibility makes the compiler more short-sighted when a particular structure is expected, like for control flow keywords requiring blocks. I don't see yet how this dynamic call approach could support such use-cases in a more user-friendly way.

But another feature can come to the rescue, the upcoming #alias directive proposed in the previous blog post. As long as the user will be willing to use this new directive, it would simply avoid these dynamic constructions, by providing enough information to the compiler to statically determine what kind of value, paths are referring to, resulting in much shorter and faster code, without the short-sightness issue.

This is the kind of problem I had to solve during object implementation and why it took much longer than planned initially.

Hope this deeper look inside the compiler's guts is not too scary. ;-) Now, back to coding for next release!

And, by the way, Merry Christmas to all Red followers. :-)

December 22, 2014

0.5.0: Objects support

We are bumping the version number up higher as we are bringing a new foundational layer and important construct to Red: object! datatype and contexts support.

Supporting objects in the Red interpreter is relatively easy and straightforward. But adding those features in the compiler has proven to be more complex than expected, especially for access-path support, paths being especially tricky to process, given their highly dynamic nature. Though, I have pushed Red beyond the edges I was planning to stop at for objects support, and the result so far is really exciting!

Object model

Just a short reminder mainly intended for newcomers. Red implements the same object concept as Rebol, called prototype-based objects. Creating new objects is done by cloning existing objects or the base object! value. During the creation process, existing field values can be modified and new fields can be added. It is a very simple and efficient model to encapsulate your Red code. There is also a lot to say about words binding and contexts, but that topic is too long for this blog entry, we will address that in the future documentation.

Object creation

The syntax for creating a new object is:
    make object! <spec>
 
    <spec>: specification block
Shorter alternative syntaxes (just handy shortcuts):
    object  <spec>
    context <spec>
The specification block can contain any valid Red code. Words set at the root level of that block will be collected and will constitute the new object's fields.

Example:
    make object! [a: 123]
    
    object [a: 123 b: "hello"]
    
    c: context [
       list: []     
       push: func [value][append list value]
    ]
You can put any valid code into a specification block, it will be evaluated during the object construction, and only then.

Example:
    probe obj: object [
        a: 123
        print b: "hello"
        c: mold 3 + 4
    ]
will output:
    hello
    make object! [
        a: 123
        b: "hello"
        c: "7"
    ]
Objects can also be nested easily:
    obj: object [
        a: 123
        b: object [
            c: "hello"
            d: object [
                data: none
            ]
        ]
    ]

Another way to create an object is to use the copy action which does not require a specification block, so does just a simple cloning of the object. Existing functions will be re-bound to the new object.

Syntax:
    copy <object>
Object access paths

In order to access object fields, the common path syntax is used (words separated by a slash character). Each word (or expression) in a path is evaluated in the context given by the left side of the path. Evaluation of a word referring to a function will result in invoking the function, with its optional refinements.

Example:
    book: object [
        title: author: none
        show: does [print [mold title "was written by" author]]
    ]

    book/title: "The Time Machine"
    book/author: "H.G.Wells"
    print book/title
    book/show
will output:
    The Time Machine
    "The Time Machine" was written by H.G.Wells
SELF reference

A special keyword named self has been reserved when self-referencing the object is required.

Example:
    book: object [
        title: author: none
        list-fields: does [words-of self]
    ]
    book/list-fields
will output:
    [title author list-fields]
Object inheritance

While cloning produces exact replicas of the prototype object, it is also possible to extend it in the process, using make action.

Syntax:
    make <prototype> <spec>

    <prototype> : object that will be cloned and extended
    <spec>      : specification block
Example:
    a: object [value: 123]
    
    c: make a [
        increment: does [value: value + 1]
    ]
    
    print c/increment
    print c/increment
will output:
    124
    125
It is also possible to use another object as <spec> argument. In such case, both objects are merged to form a new one. The second object takes priority in case both objects share same field names.
 
Example:
    a: object [
        value: 123
        show: does [print value]
    ]
    b: object [value: 99]
    
    c: make a b
    c/show
will output:
    99
Detecting changes in objects

Sometimes, it can be very useful to detect changes in an object. Red allows you to achieve that by defining a function in the object that will be called just after a word is set. This event is generated only when words are set using a path access (so inside the object, you can set words safely). This is just a first incursion in the realm of metaobject protocols, we will extend that support in the future.

In order to catch the changes, you just need to implement the following function in your object:
    on-change*: func [word [word!] old new][...]
    
    word : field name that was just affected by a change
    old  : value referred by the word just before the change
    new  : new value referred by the word
It is allowed to overwrite the word just changed if required. You can directly set the field name or use set:
    set word <value>
Example:
    book: object [
        title: author: year: none
  
        on-change*: func [word old new /local msg][
            if all [
                word = 'year
                msg: case [
                    new >  2014 ["space-time anomaly detected!"]
                    new < -3000 ["papyrus scrolls not allowed!"]
                 ]
            ][
               print ["Error:" msg]
            ]
        ]
    ]

    book/title: "Moby-Dick"
    book/year: -4000
will output:
    Error: papyrus scrolls not allowed!
Extended actions and natives for objects

You can use set on an object to set all fields at the same time. get on an object will return a block of all the fields values. get can also be used on a get-path!.

Example:
    obj: object [a: 123 b: "hello"]
    probe get obj
    set obj none
    ?? obj
    set obj [hello 0]
    ?? obj
    probe :obj/a
will output:
    [123 "hello"]
    obj: make object! [
        a: none
        b: none
    ]
    obj: make object! [
        a: 'hello
        b: 0
    ]
    hello

Find action gives you a simple way to check for a field name in an object. If found it will return true, else none.

Select action does the same check as find, but returns the field value for matched word.

    obj: object [a: 123]
    probe find obj 'a
    probe select obj 'a
    probe find obj 'hello
will output:
    true
    123
    none
The in native will allow you to bind a word to a target context:
    a: 0
    obj: object [a: 123]
    probe a
    probe get in obj 'a
will output:
    0
    123

Bind native is also available, but not completly finished nor tested.

Reflectors

Some reflective functions are provided to more easily access objects internal structure.

  • words-of returns a block of field names.
  • values-of returns a block of field values.
  • body-of returns the object's content in a block form.

Example:
     a: object [a: 123 b: "hello"]
     probe words-of a
     probe values-of a
     probe body-of a
will output:
    [a b]
    [123 "hello"]
    [a: 123 b: "hello"]
SYSTEM object

The system object is a special object used to hold many values required by the runtime library. You can explore it using the new extended help function, that now accepts object paths.
red>> help system
`system` is an object! of value:
    version          string!   0.5.0
    build            string!   21-Dec-2014/19:27:05+8:00
    words            function! Return a block of global words available
    platform         function! Return a word identifying the operating system
    catalog          object!   [datatypes actions natives errors]
    state            object!   [interpreted? last-error]
    modules          block!    []
    codecs           object!   []
    schemes          object!   []
    ports            object!   []
    locale           object!   [language language* locale locale* months da...
    options          object!   [boot home path script args do-arg debug sec...
    script           object!   [title header parent path args]
    standard         object!   [header]
    view             object!   [screen event-port metrics]
    lexer            object!   [make-number make-float make-hexa make-char ...
Note: not all system fields are yet defined or used.

Future evolutions

As this release already took a lot of time, some of the planned features are postponed to future releases. Here are a few of them.

Sometimes, it is convenient to be able to add fields to an object in-place, without having to recreate it, losing lexical binding information in the process. To achieve that, a new extend native will be added, working like originaly intended in Rebol3.

In order to help the Red compiler produce shorter and faster code, a new #alias compilation directive will be introduced. This directive will allow users to turn an object definition into a "virtual" type that can be used in type spec blocks. For example:
    #alias book!: object [
        title: author: year: none
        banner: does [form reduce [author "wrote" title "in" year]]
    ]
    
    display: func [b [book!]][
        print b/banner
    ]
This addition would not only permit finer-grained type checking for arguments, but also help the user better document their code.

Another possible change will be in the output mold produces for an object. Currently such output will start with "make object!", this might be changed to just "object", in order to be shorter and easier to read in addition to be more consistent to the way function! values are molded.

Fixed issues

In order to make this release happen as quickly as possible, we have not fixed all the open tickets that were planned to be fixed in this release, but we still managed to fix a few of them. The other pending tickets will be fixed in the upcoming minor releases.

I should also mention that 537 new tests were added to cover objects features. The coverage is already good, but we probably need more of them to cover edge cases.

That's all for this blog article! :-)

I will publish another blog entry about additional information regarding the implementation strategy used by the compiler for supporting contexts and object paths.

As we have almost completed other significant features during the last months, you should expect new minor releases happening very quickly in the next weeks. They will include:

  • New cross-platform console engine written entirely in Red (no dependencies).
  • New Android toolchain for creating APK files 100% ported to Red (no dependencies).
  • Full error and exceptions support at Red level.
  • Redbin initial implementation (not started yet).

Also, the work for 0.6.0 has started already (GUI support), even if its at prototype stage right now. I plan to release a first minimal version in the next few weeks (we will extend it step by step until 1.0).

Hope the waiting for the new release was worth it. ;-)

August 3, 2014

0.4.3: Floating point support

After a long time having only partial floating point support in Red/System, it comes now to Red with a broader support, thanks to the terrific work from Qtxie and Oldes, who managed to push Red further while I was busy moving from Europe to China (though, this might be the topic for another blog entry, as requested by many of you).

The new float! datatype implements IEEE-754 64-bit floating point format. It is available with most of the usual math functions support:

  • infix operators: +, -, *, /, **.
  • prefix base functions: add, substract, multiply, divide, power.
  • trigonometric functions:  cosine, sine, tangent, arcsine, arccosine, arctangent.
  • other math functions: log-2, log-10, log-e, exp, square-root, round

Note that these trigonometric functions are taking arguments in degrees, a /radians refinement is provided for input values in radians. However, this can result in extra verbosity for some long math expressions where using only radians, like:
((sine/radians b) * (cosine/radians c)) + ((cosine/radians b) * (sine/radians c))
Some radians-oriented shortcuts to these functions are also provided for convenience: cos, sin, tan, arcsin, arccos, arctan. So the above expression becomes:
((sin b) * (cos c)) + ((cos b) * (sin c))
Here are some code examples from Red console:
red>> 1.23456
== 1.23456
red>> 1e10
== 10000000000.0
red>> 1.23 * 2
== 2.46
red>> 1.23 * 2.0
== 2.46
red>> to integer! 1.23 * 2.0
== 2
red>> cos pi
== -1.0
red>> sin pi
== 0.0
red>> cos pi / 2
== 0.0
red>> cos pi / 3
== 0.5
red>> cosine/radians pi / 3
== 0.5
red>> cosine 60
== 0.5
red>> .1 + .2 + .3
== 0.6
red>> .1 + .2 + .3 = .6
== true
red>> .1 + .2 + .3 - .6
== 1.110223024625157e-16
red>> float? load "0.1"
== true
red>> to float! 1
== 1.0
red>> 1 = to integer! to float! 1
== true
As you can see, Red tries to give you meaningful outputs even when the result is not exact, but this approach has its limits too. Qtxie has ported partially dtoa() functions to Red/System, however, the overhead of the additional code (20-40KB once compiled) is quite costly given how tiny is currently our runtime library (~350KB). So, for now, that implementation has been provided as an optional library for Red/System, and will be modularized for Red, once modules will be supported.

IEEE-754 special values

You might know that standard floating point format supports a few extra special values that are meant to make some calculation possible in edge cases. Those are also supported natively by Red, with the following literal formats:
Not a Number (NaN)        :  1.#NaN
Positive Infinity (+INF)  : +1.#INF (or just 1.#INF)
Negative Infinity (-INF)  : -1.#INF
Positive Zero             : +0.0 (or just 0.0)
Negative Zero             : -0.0
These values are mostly intended for scientific calculations, you do not have to worry about them. They can be produced as results of some math operations on floats, but by default, an error will be thrown instead.

In case, you need to operate with maximum precision, and have all the special float values as results instead of errors, a couple of flags are available for that through the system special access. The syntax is:
system/float-options [spec]

[spec]: block of flags (word! | set-word!) with values (logic! | word!)
Valid flags are:

  • pretty?: enables pretty printing of float numbers when very close to an integer value (default: true)
  • full?: enables math operations on float special values (default: false)

 Examples:
red>> 4.000000000000001e32
== 4.0e32
red>> system/float-options [pretty?: no]
red>> 4.000000000000001e32
== 4.000000000000001e32
Armhf support

So far, Red supported only the armel ABI for ARM backends. Since this release, we fully support now armhf ABI too, through a specific compilation option that can be found in the new RPi compilation target (intended mainly for default OS on RaspberryPi). The main difference between these ABI is the way float values are passed as arguments to functions, armel requires passing them on stack, while armhf requires passing them through FPU registers.

Other changes

  • url! datatype preliminary support: all actions are working, but no path access support yet.
  • New actions: reverse, random, swap, take, to(*), trim
  • New natives: same?, NaN?
  • New mezzanines: float?, routine?
  • Red/System FPU direct access through system/fpu/* options.
  • Help command now displays full help on routines too.
  • Many bug fixes and a few wishes granted.

(*) to is currently limited to integer/float/string conversions only.

What's next?

After the digression in the floating point lands, we go back to our main roadmap, so in the next releases, expect (in no particular order):

  • GUI support for Android / Windows platforms
  • Improved toolchain for Android APK generation
  • Object compilation support
  • New console engine
  • Error! datatype and exceptions handling
  • Typeset! and other new datatypes
  • Redbin format specification and implementation for the compiler
  • Improved compiler performance

Thanks for your patience and support during these last months, we are now back to our cruise development speed, so expect faster changes until the end of the year.

April 29, 2014

0.4.2: Unicode console and FreeBSD support

This long awaited new release is now available. As I have been travelling a lot in the last months, this release has been delayed much more than I wanted. Anyway, we managed to achieve a really big amount of work, as shown by the 500+ commits since previous release and the 75 fixes over 210 new tickets opened. As usual, we strive to keep the number of opened tickets (especially bug reports) as low as possible, achieving 97.5% of closed tickets out of a total of 794 tickets so far! We really do care about bug reports.

New runtime lexer

The first runtime lexer (wrapped by load function) was implemented a year ago, as a quick hack for the console addition to Red. It was coded in Red/System and supported ASCII inputs only. It was not meant to stay more than a few weeks, but as often in the software world, temporary code lifespan exceeds by far the wishes of the author. The addition of Parse dialect in previous release has opened the possibility of rewriting the Red runtime lexer using the Unicode-aware parse function. It turned out to be a great design choice and opens even more interesting future options like I/O streaming support (when parse will support it) or dynamically extending the lexical rules (when loading custom datatypes for example).

Improved console

The new runtime lexer is now powering the Red console, so we finally have proper Unicode input support!

A help system has also been provided, including the following functions: help, what and source. Try them from the console!

From the console code, the line editing features have been extracted in a different source file that can be now included in your Red programs when you need user input support. For that, two new functions have been provided (working like in Rebol): input and ask.

Moreover, a new branch was started in order to provide cross-platform line editing capabilities without the burden of external dependencies that have proved to be problematic and limited. The new vt100 version should work fine, but it is unfinished. Contributors with deep terminal coding experience are welcome to help improve the current code. We are aiming at a cross-platform console engine that could be used both in CLI and GUI apps.

Additional functions
  • New actions: absolute, remainder, complement, power, odd?, even?, and, or, xor, reverse
  • New natives: complement?, positive?, negative?, min, max, shift, to-hex
  • New operators: <<, >>, >>>
A new option was added to the system function: system/interpreted?, that will return true if the code is run by the interpreter (remember that do invokes the interpreter until we get JIT-compilation abilities).

Parse and load have been extended to accept a /part refinement.

Infix operators can now be created from existing functions or routines.

A first call function implementation has been contributed by Bruno Anselme with support for both Red and Red/System.

FreeBSD support

Yes, we got it now! :-) All thanks to Richard Nyberg who kindly provided us with the low-level patches required to make Red work on FreeBSD.

Red/System changes

The Red/System lexer used to be simply the load native from Rebol, which was a great way to speed up the development at the beginning, but was also limitating the syntax to only what Rebol2 accepts. Now the Red/System lexer uses the same code as the Red lexer (the compiler version, not the runtime one), freeing the Red/System syntax from the limitations and making it truly a dialect of Red!

Literal arrays support has been added also in order to facilitate initialization of arrays of value (until we get a first class array! datatype).

CPU registers read/write access has been added. It will be extended in the future to support special registers (like status flags register).

The maximum number of function local variables supported by Red/System was limited to 16, this was limitating also the number of local words that could be used in a Red function. This limitation has now been raised much higher, at least 512 local variables are now allowed.

Work in progress...

Object support is already present in this release, but is it not official yet, as it is supported by the interpreter only and not the compiler. Expect quick progress on this front.

The Android GUI support is also under heavy work in the android branch. In order to implement a proper GUI API, the implementation of a VID-like dialect has started, with Android as first back-end. Windows support should follow shortly, then Linux (most probably using GTK+) and finally MacOSX (once we implement the objective-c bridge).

Gear second!

I am not made of rubber, but I can go gear second too! ;-) You may have not noticed, but the project is rapidly growing up in the last months. It is moving faster and on a larger scale as more contributors are joining. We also get better organized. This is the github stats for just this month:


The most important power-up we got was the addition of Xie Qingtian (aka qtxie) to the Red core team. Xie is an amazingly skilled young programmer from China, who is contributing to Red since more than a year now. But the thing is that he is working full time on Red project now, thanks to the kind sponsoring of CSDN and his CEO, Jiang Tao. Xie is the one who implemented all the new functions listed above and in a short amount of time! So consider that from now on, Red will be advancing twice faster thanks to him! ;-)

In order to organize better the work on Red, we are now using extensively Trello as our main task manager tool. The Red tasks board contains three main lists:

  • "Work in progress": for features we are working on.
  • "Road to 1.0": lists the required features for 1.0 version.
  • "Milestones": helps us organize upcoming releases.

Last but not least, the number of visitors on this site and the github repo has, at least, doubled since new year, thanks to an article on CSDN about Red, our Google Summer of Code campaign and the successful StackOverflow ad campaign (finished earlier this month) run by HostileFork, that brought us more than 10k new visitors who clicked the ad, making it the most clicked ad on SO since the new year! The ad is still visible here.

Big thank to all the people that have contributed to this (big) release. Enjoy it! :-)

February 6, 2014

Project Ideas for the Google Summer of Code 2014

Red has been ramping up its recruiting efforts in 2014.  So far in January, we've done very well with our open-source recruitment campaign on StackOverflow.  Now in February, we are throwing our hat into the ring to be a mentoring organization for Google's Summer of Code:


If you haven't heard about Summer of Code, it pairs college students with mentors who represent an open source project.  The students work remotely and are paid a stipend... and they are expected to complete a measurable goal in the 3 months that they have.  There's an opportunity to start getting involved before the summer, and hopefully the students will have a good experience and stay involved after the summer ends!

Though Nenad will be too busy in the new Beijing Red headquarters :-) to mentor, several members of the Red team have stepped up to volunteer if we are selected.  There is of course around-the-clock support in the [Rebol and Red] chat room, but Google needs commitments from specific people to fill out paperwork and reviews.  As the project "admin" Nenad should only have to be consulted if there was a disagreement between the student and the mentor (which would not happen!)  Of course he would be reviewing any pull request, as with any other code submission.

Google says the best project ideas come from students, which sounds like a good idea.  Still, they require a "starter list" of example projects to originate from the potential mentoring organizations.  So here is the one we came up with...