March 25, 2016

0.6.0: Red GUI system

Five years ago, when I started writing the first lines of code of what would become later the Red/System compiler, I had a pretty good picture already of what I wanted to achieve with Red, and all the ideal features that should be included, just not sure how much time and efforts it would require to have them. Two years and half ago, baby Red printed its first output. And today, we celebrate a major step forward with the addition of a brand new GUI system entirely written in Red itself! What a journey!



Here it is, the long awaited 0.6.0 release with its massive 1540 commits! The major new features are:
  • View engine, with Windows backend (from XP to 10)
  • VID dialect
  • Draw dialect
  • Reactive GUI programming
  • GUI console
  • Simple I/O support for files and HTTP(S) queries.
  • Codecs system with following codecs available: BMP, GIF, JPEG, PNG
  • Objects ownership system

All those additions made our Red executable grow from 767 KB to 910 KB (Windows platform), sorry for the extra 143 KB, we will try to [red]uce that in the future. ;-)

Let's start with the elephant in the room first, the GUI system. Here is an architecture overview:

Only the Windows backend is fully usable for now, Android and OS X are work-in-progress, Linux (using GTK) will follow soon, iOS will come later this year. Also, we have other targets in mind, like JS/HTML which are not scheduled yet, but could come this year too.

Red/View

First let me mention that View, VID and Draw were invented by Carl Sassenrath (of Amiga fame) in the Rebol language, a long time ago. Red's version retains all the best features and pushes the boundaries of simplicity even further. The main features of our View engine are:
  • A live updating mode that reduces the need to a single view function in most cases.
  • Full abstraction over rendering backends.
  • Two-way binding using live objects.
  • Event bubbling/capturing stages.
  • Built-in drag'n drop support for most face types.
  • Gestures support (experimental).
  • Native widgets support.
  • Full integration with the OS features.
  • Flexible backend support that can be mapped to virtually any kind of UI library.

The current list of supported widgets is: base, text, button, check, radio, field, area, text-list, drop-list, drop-down, progress, slider, camera, panel, tab-panel, group-box.

Next releases will bring more widgets, like: table, tree, divider, date/time picker, web-view and many others!

For more info about View, see the View reference document.

Main differences between Red/View and Rebol/View are:
  • Red relies on native widgets, Rebol has custom ones only, built over a 2D vector library.
  • Red faces are synchronized with their widgets on display in realtime by default, Rebol faces require manual calls to many functions for keeping faces and widget updated.
  • Red introduces reactive GUI programming.

Red/View will update both face and graphic objects in realtime as their properties are changed. This is the default behavior, but it can be switched off, when full control over screen updates is desirable. This is achieved by:
    system/view/auto-sync?: off
When automatic syncing is turned off, you need to use show function on faces to get the graphic objects updated on screen.

VID dialect

VID stands for Visual Interface Dialect. It is a dialect of Red which drastically simplifies GUI construction. VID code is dynamically compiled to a tree of faces, feeding the View engine. You can then manipulate the face objects at runtime to achieve dynamic behaviors. VID offers:
  • Declarative programming.
  • Automatic layout system.
  • Cascading styles.
  • Default values for...everything.
For more info about VID, see the specification.

In case you are reading about Red or Rebol for the first time, here are a few code demos to show how simple, yet efficient, is our approach to GUI programming:
    ;-- GUI Hello word
    view [text "Hello World"]
    
    ;-- Say Hi to the name you type in the field
    view [name: field button "Hi" [print ["Hi" name/text]]]
    
    ;-- Demo simple reactive relations, drag the logo around to see the effect
    view [
        size 300x300
        img: image loose http://static.red-lang.org/red-logo.png
        return
        shade: slider 0%
        return
        text bold font-size 14 center "000x000" 
            react [face/text: form img/offset]
            react [face/font/color: white * shade/data]
    ]
    
    ;-- Simple form editing/validating/saving with styles definitions
    digit: charset "0123456789"
    view [
        style label: text bold right 40
        style err-msg: text font-color red hidden
    
        group-box "Person" 2 [
            origin 20x20
            label "Name" name: field 150
            label "Age"  age:  field 40
            label "City" city: field 150
            err-msg "Age needs to be a number!" react [
                face/visible?: not parse age/text [any digit]
            ]
        ]
        button "Save" [save %person.txt reduce [name/text age/text city/text]]
    ]
    set [name age city] load %person.txt
    ?? name ?? age ?? city

You can run all those examples by copy/pasting them one-by-one into the Red console for Windows. To get the console, just download it and double-click the Red binary, wait ~20 seconds for the console to be compiled for your OS (yes, that little file contains the full Red toolchain, runtime library and console source code), paste the code and have fun. ;-)

Draw dialect

Draw is a 2D vector-drawing dialect which can be used directly, to render on an image, in faces for local rendering, or specified through VID. It is still a work in progress as not all features are there yet. We aim at full Rebol/Draw coverage and full SVG compatibility in the not-too-distant future.

A simple example of Draw usage:
    shield: [
        fill-pen red   circle 50x50 50
        pen gray
        fill-pen white circle 50x50 40
        fill-pen red   circle 50x50 30
        fill-pen blue  circle 50x50 20
        
        pen blue fill-pen white
        polygon 32x44 45x44 50x30 55x44 68x44 57x53 60x66 50x58 39x66 43x53
    ]
    
    ;-- Draw in a draggable face, in realtime.
    view [
        size 300x300
        img: image 100x100 draw shield loose
        at 200x200 base white bold react [
            [img/offset]
            over?: overlap? face img
            face/color: get pick [red white] over?
            face/text: pick ["Hit!" ""] over?
        ]
        button "Hulk-ize!" [replace/all shield 'red 'green]
        button "Restore"   [replace/all shield 'green 'red]
    ]
Copy/paste the above code example in a Red console on Windows, and become an Avenger too! ;-)

For more info about Draw, see the specification.

Main differences between Red/Draw and Rebol/Draw:
  • Red does not yet cover all the commands of Rebol/Draw yet.
  • Red's version allows commands to be grouped in blocks, ease-ing insertion/removal at run-time.
  • Red's version allows commands to be prefixed with a set-word, allowing to save local position in Draw blocks in a word.

Reactive GUI programming

This is a deep topic which should be part of a future separate blog article. So, I will just copy/paste here the little information already in the VID documentation:

Reactions (or reactors, not sure yet which terms is the most accurate) are created using the react keyword, directly from Red code or from VID dialect. The syntax is:
    react [<body>]
    
    <body> : regular Red code (block!).
This creates a new reactor from the body block. When react is used in VID, as a face option, the body can refer to the current face using face word. When react is used globally, target faces need to be accessed using a name.

Reactors are part of the reactive programming support in View, whose documentation is pending. In a nutshell, the body block can describe one or more relations between faces properties using paths. Set-path setting a face property are processed as target of the reactor (the face to update), while path accessing a face property are processed as source of the reactor (a change on a source triggers a refresh of the reactor's code).

Basically, it is about statically-defined relations between faces properties, without caring when or how the reactive expressions will be evaluated. It will happen automatically, when needed. Here are a couple of examples you can copy/paste in the Red console on Windows:

Make a circle size change according to slider's position:
    view [
        sld: slider return
        base 200x200 
            draw  [circle 100x100 5]
            react [face/draw/3: to integer! 100 * sld/data]
    ]

Change the color of a box and a text using 3 sliders:
    to-color: function [r g b][
        color: 0.0.0
        if r [color/1: to integer! 256 * r]
        if g [color/2: to integer! 256 * g]
        if b [color/3: to integer! 256 * b]
        color
    ]

    to-text: function [val][form to integer! 0.5 + 255 * any [val 0]]

    view [
      style txt: text 40 right
      style value: text "0" 30 right bold
    
      across
      txt "Red:"   R: slider 256 value react [face/text: to-text R/data] return
      txt "Green:" G: slider 256 value react [face/text: to-text G/data] return
      txt "Blue:"  B: slider 256 value react [face/text: to-text B/data]
    
      pad 0x-65 box: base react [face/color: to-color R/data G/data B/data]
      return
    
      pad 0x20 text "The quick brown fox jumps over the lazy dog." font-size 14
        react [face/font/color: box/color]
    ]


GUI console

We have a GUI console now, in addition to the existing CLI one!

The GUI console is now the default on Windows platform, and is fully Unicode-aware. The system shell (DOS) console is still available using --cli option:
    red --cli
The GUI console is still in its infancy and will be enhanced a lot in future releases. Anyway, so far, it already supports:
  • history of commands
  • completion on words and object paths
  • multi-line editing for blocks, parens, strings, maps and binaries.
  • navigation using HOME and END keys
  • select/copy/paste using the mouse and keyboard shortcuts
  • auto-scrolling when selecting with the mouse out of the boundaries
  • very fast text rendering
  • automatic vertical scroll bar
  • customizable prompt
Try this cool one-liner for making the prompt more active:
    system/console/prompt: does [append to-local-file system/options/path "> "]

This is how the GUI console looks like:


Simple I/O support

In order to have really some fun with the GUI, we have added some minimal support for blocking IO basic actions covering files and HTTP(S) requests. read and write action are available now. Their /binary, /lines and /info refinement are working. do, load, save have also been extended to work with files and urls.

When not using /binary, read and write are expecting UTF-8 encoded data. Support for ISO8859-1 and other common encoding formats will be available in next release.

The full IO will come in 0.7.0 with ports, full networking, async support and many more features.

Codecs

Codec system support has made his entrance in this release. It is a very thin layer of encoders/decoders for binary data, integrated with load, save and actions which rely on /as refinement. load and save will auto-detect the required encoding format and try to apply the right encoder or decoder on the data.

Currently only image format codecs are provided: BMP, PNG, GIF, JPEG. Any kind of encoding (related to IO) is a good candidate for becoming a codec, so expect a lot of them available in the future (both built-in Red runtime and optionaly installable).

For example, downloading a PNG image in memory, and using it is as simple as:
    logo: load http://static.red-lang.org/red-logo.png
     
    big: make font! [name: "Comic" size: 20 color: black]
    draw logo [font big text 10x30 "Red"]
    view [image logo]
Saving a downloaded file locally:
    write/binary %logo.png read/binary http://static.red-lang.org/red-logo.png
Saving images is not fully functional yet, PNG should be safe though.

Objects ownership system

Red's objects ownership system is an extension of object's event support introduced in previous releases. Now, an object can own series it references, even nested ones. When an owned series is changed, the owner object is notified and its on-deep-change* function will be called if available, allowing the object to react appropriately to any change.

The prototype for on-deep-change* is:
    on-deep-change*: func [owner word target action new index part][...]
The arguments are:
  • owner: object receiving the event (object!)
  • word: object's word referring to the changed series or nested series (word!)
  • target: the changed series (any-series!)
  • action: name of the action applied (word!)
  • new: new value added to the series (any-type!)
  • index: position at which the series is modified (integer!)
  • part: number of elements changes in the series (integer!)
Action name can be any of: random, clear, cleared, poke, remove, removed, reverse, sort, insert, take, taken, swap, trim. For actions "destroying" values, two events are generated, one before the "destruction", one after (hence the presence of cleared, removed, taken words).

When modifications affect several non-contiguous or all elements, index will be set to -1.
When modifications affect an undetermined number of elements, part will be set to -1.

Ownership is set automatically on object creation if on-deep-change* is defined, all referenced series (including nested ones), will then become owned by the object. modify action has been also implemented to allow setting/clearing ownership post-creation-time.

As for on-change, on-deep-change* is kept hidden when using mold on object. It is only revealed by mold/all.

Here is a simple usage example of object ownership. The code below will create a numbers object containing an empty list. You can append only integers to that list, if you fail to do so, a message will be displayed and the invalid element removed from the list. Moreover, the list is always sorted, wherever you insert or poke a new value:
    numbers: object [
        list: []
    
        on-deep-change*: function [owner word target action new index part][
            if all [word = 'list find [poke insert] action][
                forall target [
                    unless integer? target/1 [
                        print ["Error: Item" mold target/1 "is invalid!"]
                        remove target
                        target: back target
                    ]
                ]
                sort list
            ]
        ]
    ]
    
    red>> append numbers/list 3
    == [3]
    red>> insert numbers/list 7
    == [3 7]
    red>> append numbers/list 1
    == [1 3 7]
    red>> insert next numbers/list 8
    == [1 3 7 8]
    red>> append numbers/list 4
    == [1 3 4 7 8]
    red>> append numbers/list "hello"
    Error: Item "hello" is invalid!
    == [1 3 4 7 8]
    red>> numbers
    == make object! [
        list: [1 3 4 7 8]
    ]
Object ownership is deeply used in Red/View, in order to achieve the binding between face objects and the widgets on screen, and the automatic "show-less" synchronization. 

The work on this is not yet completed, more object events will be provided in future releases and the ownership support extended to enable objects to own more datatypes. More documentation will be provided once the work on that will be finished. In the future, its use will be extending to other frameworks and interfaces. Such "reactive objects" will be called "live objects" in Red's jargon.

Red/System changes
  • Full stack traces in debug mode on runtime errors.
  • New compilation directive: #u16 (literal UTF-16LE strings support).
  • Added log-b native function for getting the binary logarithm of an integer.
  • Added equal-string? runtime function for testing c-string! equality.
  • Several improvements to some compiler errors reporting accuracy.
  • Improved function! type support.
  • New compilation option: debug-safe? (for safer stack traces)
  • New --catch command-line option for console to open on script errors.
  • Improved compilation speed of variables assignment.
  • Fixes for broken exceptions support on ARM backend.

Additions to the Red runtime library

New functions

  • show, view, unview, draw, layout, react, size-text, to-image, do-events, dump-face, within?, overlap?, remove-reactor, set-flag, find-flag?, center-face, insert-event-func, remove-event-func.
  • event?, image?, binary?.
  • debase, wait.
  • request-file, request-dir.
  • read, write, exists?, to-local-file, to-red-file, dirize, clean-path, split-path.
  • what-dir, change-dir, list-dir.
  • also, alter, extract, collect, split, checksum, modify, unset.
  • as-color, as-rgba, as-ipv4.
  • cd, ls, ll, pwd, dir. (console-only)
Use help in the console to get more information about each function.

New datatypes

  • binary!
  • event! (Windows only for now)
  • image! (Windows only for now)

Binary! datatype supports all the series actions. Literal base 2, 16 and 64 encodings are available:
    red>> 2#{11110000}
    == #{F0}
    red>> to string! 64#{SGVsbG8gV29ybGQh}
    == "Hello World!"
Event! and image! are a work-in-progress, though image! is already very capable (documentation will be added soon).

Other changes

set and get native improvements:

If A and B are object values, set A B will now set the values in A from B, for the fields they have in common (regardless of the fields definition order in the objects).

Added /only and /some refinements:
  • /only: set argument block or object as a single value
  • /some: `none` values in the argument block or object are not set

o Icons and other "resources" are now supported for inclusion in Windows executables. They can be set from Red's main script header, these are the currently supported options:

  • Icon: file! or block! of files
  • Title: string!
  • Author: string!
  • Version: tuple!
  • Rights: string!
  • Company: string!
  • Notes: string!
  • Comments: string!
  • Trademarks:  string!

If no Icon option is specified, a default Red icon will be provided.

o index? action is now allowed on words. It will return the word's index inside a context or none if the word is unbound. This is a shortcut for the following idiom:
    index? find words-of <object> <word>
 
o Remaining list of changes:

  • Implemented type-checking for infix operators in the interpreter.
  • Implemented native! functions type-checking support when called by compiled code.
  • Added system/state/trace? for enabling/disabling call stack traces on errors.
  • system/options/args gets the command-line string.
  • Added DO/ARGS support.
  • Error report for catchable infinite block rules recursions in Parse.
  • Added limits to Parse stack to avoid eating up all the memory.
  • Auto-conversion of float values in routines.
  • Big series (> 2MB) support enabled.
  • Lexer support for base2 and base64 encoding.
  • DO and LOAD work on file! and url! values now.
  • Added support for cycles detection for MOLD/FORM and comparisons.
  • Support for set operations on hash!.
  • SORT works on paren! now.
  • string! to issue! conversion support.
  • file! to string! conversion support.
  • Allowed float! values as arguments to AS-PAIR and MAKE pair!.
  • Added percent! support in vector! series.
  • Added matching typesets support to Parse.
  • Added PUT support to object! and any-series!.
  • Added support for make bitset! <binary>
  • Setting a tuple component to none now eliminates the component.
  • Support for HOME and END key in console.
  • Multiline editing support for paren! and map! in console.
  • Added proper error handling for malformed paths evaluation attemps.
  • Scripts using routines will now output a proper error when run from interpreter.
  • Better error handling when decoding UTF-8 string.
  • Allow PROBE to have an unset! value as argument.
  • Support X in addition to x for pair! literal syntax.
  • Prevent empty conditions in conditional iterators from entering an infinite loop.
  • Improved formatting of error messages arguments.
  • Several output improvements to HELP.
  • Allow DIR? to take an url!.
  • Allow system/console/prompt to be an active value (e.g.: a function).

Ticket processing

We have closed 260+ tickets since last release (0.5.4), among which 54 concern issues in previous releases. We have currently ~92.5% closed tickets overall, which is a bit lower than the usual 95%+, mostly due to the huge amount of new code and feature in this release. So, we will aim at getting back to a lower number of opened tickets for the next release.

I would like to make a big thank to all the contributors who reported issues, tested fixes and sent pull requests. It has been, more than ever given the number of newly implemented features, a huge help in making Red better. I would like to especially thank namely a few people for their outstanding contributions:

  • WiseGenius: for helping us solve the epic crush library generation bug, improvement suggestions and huge work on testing/reporting GUI issues! 
  • nc-x: for help in testing the GUI, making many useful issue reports and improvement suggestions.
  • The "Czech group" (Pekr, Oldes, Rebolek): for their constant support and for taking care of the Red community when I'm not available. ;-)
  • PeterWAWood: for bringing us the ~30'000 unit tests, testing framework and constant help and support, since day one!
  • Micha: for issues reporting and kindly providing us an online Mac OSX server for our build farm.

What's next?

Our focus for next releases (0.6.x) will be:
  • Drastically speed up compilation time by pre-compiling the runtime library.
  • Simple garbage collector integration.
  • Improvement of our Windows GUI backend.
  • First usable versions of MacOSX and Android GUI backends.
  • Integration of our Android APK building toolchain in master branch.
  • Improvements for reactive GUI programming support.
  • Custom widgets creation framework.
  • Animation and timers support.
  • More documentations and tutorials for beginners.
  • More code demos.

See the detail for next releases on our public Trello board and come to our chat room to ask any question.

In the meantime, enjoy the new Red, I hope to see many impressive GUI demos and apps in the next weeks. ;-)

December 19, 2015

Answers to community questions

Features, future directions

Will Red get lisp like macros?

The short answer is: yes, but.... We first need to define how we want the API for macros to look like and how to prevent users from shooting themselves in the foot too easily using them. Also, macros are challenging for a visual debugger, we also need first to have a clear vision of how an IDE would handle that (I have not yet looked into how top Lisp IDE managed that, though I hope they have solved it elegantly).

About the ETA, we had a strong need for macros in our Android bridge, though that need might be gone now that Red's startup time has been vastly improved earlier this year. Answer to that should come in a few weeks, once we merge the existing Android backend with our new GUI engine.

When will Red get optional types?

It already has, along with multi-typing: you can specify one or more allowed types for function arguments, return value and local words. Only arguments type-checking is implemented for now, the rest will come on the path to 1.0. The most interesting part of optional typing is how much the compiler will be able to leverage it to generate much faster code when everything is mono-typed. Though, we will find that out after 1.0, once we will work on the self-hosted compiler and toolchain. As the current toolchain has a limited lifetime, we try to avoid any non-mandatory feature before 1.0.

Will Red have built-in support for some type of concurrency and/or parallel processing? What kind of model is it going to follow?

Certainly. Increase in computation power is now horizontal, with multiple cores, multiple processors and distributed architectures, so, strong concurrency support is a key part of a modern general-purpose language.

For now, the model we aim at is the Actor model. It is a good fit for Red and would provide a clean way to handle concurrent and parallel tasks across multiple cores and processors. Though, a few years has passed since that plan was made, so we will need to revisit it when the work on 0.9.0 will start, and define what is the best option for Red then. One thing is sure, we do not want multithreading nor callback hell in Red. ;-)

Do you plan to implement any kind of app/web server in Red, similar to Cheyenne available for Rebol?

As Cheyenne's author, I have strong plans for a new web application server with much better scalability than what you could achieve using Rebol. Red 1.0 should come with pretty strong server-side programming abilities out of the box, then on top of that, we'll provide a modern framework for webapp creations (think GWT or Opalang-like approach).

In addition to that, we'll have a Cheyenne RSP compatibility layer for running old Cheyenne scripts aiming at at drop-in replacement for existing webapps.

Will Red support multiselect/switch soon?

As soon as possible, maybe for the upcoming 0.6.0 release.

Will we get promises/futures in Red?

Possibly. We will experiment with that in one of the 0.7.x releases. We will have to see how such abstraction could integrate in our future concurrency model.

Will Red get direct access to Android's (and IOS later) camera, location, gyroscope, etc features?

Absolutely, our GUI engine already features a camera widget (in our Windows backend). The work on Android backend in 0.6.x version will bring wrappers to all the common hardware features.

Red is going to get modules support in future, what about Red/System?

As Red/System is an embedded dialect of Red, Red's upcoming modules system will allow inclusion of Red/System parts, so a separate modules system for it is not necessary for now.

Will function! be first-class datatype in Red/System v2?

Strictly speaking no, as you won't be allowed to create new functions from Red/System at run-time (but you will be able to create new Red/System functions from Red dynamically). The other first-class features will be possible (to a greater extent than today): passing function! pointer as arguments, returning a function! value from a function or assigning a function! pointer to a variable.

Will Red have the equivalent of Go lang's net package?

Red will feature a complete networking layer in 0.7.0, including async IO support, through a nice high-level API (similar to Rebol's one). So DNS, TCP, UDP and many more common protocols will be built-in, fortunately, relying on a very lightweight API, unlike Go's net package. ;-)

What about a package manager (in future)?

We have a modules system planned for 0.8.0. Design details are not yet defined, though we'll strive to integrate the best ideas from other existing package managers around.

Is there going to be inbuilt unit testing, something like http://golang.org/pkg/testing/?

We'll have a built-in unit testing support, probably starting with a lightweight one integrated into our upcoming modules system.

Is there a chance Red gets self-hosted sooner than initially planned, removing the R2 dependency?

Self-hosting Red means rewriting the toolchain (compilers, linker and packagers) in Red (currently written in Rebol2). Technically, 0.6.0 should have all the needed features for starting such rewrite, unfortunately, we currently don't have the resources to start such big task while continuing the work towards 1.0. The self-hosting work would not be a port of the current toolchain, it would use a very different architecture (because of JIT-compilation requirement and extra features of Red compared to Rebol). We aim at a programmable modular toolchain model, not very far from LLVM (just simpler and magnitudes smaller).

To be accurate, fully removing Rebol2 dependency is a two steps process:

  1. Removing the need for Rebol/SDK to build the Red binary, making it easy for anyone to rebuild Red binary from sources.
  2. Rewriting the toolchain in Red.


Developers, community, documentation

How do you regard the development of Red 2.0 to proceed in terms of speed/progress? Will it be faster or equal to current road to Red 1.0?

Red 2.0 is mostly about rewriting the toolchain in Red, which represents only 25% of the current Red codebase (the other 75% part is the runtime library). Moreover, the modular architecture and public API  of the new toolchain will make it much easier to write and integrate contributions from third-parties, so we'll gear all our efforts towards involving as many skilled contributors as possible. If you want Red 2.0 to come quicker, helping Red's user base grow up by contributing, writing apps, docs and tutorials is the best thing you can do right now. ;-)

What do you think is the "killer app" Red should provide, in order to attract more of developers/newcomers?

Definitely an innovative IDE. ;-) Beyond that, I believe that a successful Android app written in Red could do a lot to spread Red usage widely. If you have a great idea for such app, you'll soon be able to code it in Red 0.6.1, with full Android support.

What about documentation comments (something like rustdoc https://doc.rust-lang.org/book/documentation.html)?

In Red, like in Rebol, docstrings are allowed in functions (and in modules once we have them), so they can be processed more easily than comments (which exist only when the source is in text form). That's one of the tangible advantages of having an homoiconic language. That is also how the help command works in the console, it extracts information at runtime from functions and the environment.

That said, if you want heavier documentation embedded inside your code, you can easily define your own format for that and writing a preprocessor for it should be almost trivial using our Parse dialect (either in text form or after loading, in block form). You can go as far as implementing a Literate Programming layer if that suits your taste, like this one made for Rebol.


That's all for this time, if you want to discuss some parts deeper, you are welcome to join our chat-room on Gitter, which is way more convenient than Blogger's comment system.

See you soon for the 0.6.0 release, don't miss it! ;-)


June 12, 2015

0.5.4: New datatypes, exceptions and set operations

This new version turned out to be a major release, given the vast number of new features which made their way into it. Hope it was worth the waiting. ;-)

In preparation for the GUI support and the DSL that will come with it, a new range of datatypes has been implemented.

Pair! datatype

A pair is a couple of integer! values used to represent mainly dimensions and coordinates. Its literal representation separates the two integers by an `x` character.
    1x2
    -56x1234
    -3x-8
Pair! is a scalar datatype that supports math and bitwise operations.
    1x2 + 2x3
    == 3x5

    3x4 * 2x3
    == 6x12

    10x10 * 2
    == 20x20

    60x10 % 4
    == 0x2
To access pair elements, path syntax provides a handy way. The left value is referenced by `x` or just 1, the right value is referenced by `y` or just 2.
    a: 5x6
    a/x
    == 5
    a/y
    == 6

    b: a + 10x1
    b/1
    == 15
    b/2
    == 7
It also works for modifying pair elements.
    a: 5x6
    a/x: 0
    a/2: -1
    a
    == 0x-1
A pair! can be constructed using different approaches, depending on how pair elements are provided (as separate values or as a block).
    as-pair 3 4
    == 3x4

    make pair! [4 5]
    == 4x5

    make pair! 10
    == 10x10
The following notable actions are supported on pairs: random, absolute, add, divide, multiply, negate, remainder, subtract, and, or, xor, pick, reverse.

Pair! will be extended in the future to handle float values, in addition to integers.

Percent! datatype

Percent! provides an elegant way to represent numbers as fractions of 100 using a natural syntax:
    20%
    -45.285%
    0.1%
    100%
As it is implemented using a 64-bit float internally, it supports all the same math actions and operations as float! values, namely: random, absolute, add, divide, multiply, negate, power, remainder, round, subtract.
    20 * 80%
    == 16
    33,33% * 250
    == 83.325
Tuple! datatype

The tuple! datatype is a finite list of 3 up to 12 bytes. It offers a versatile way to represent different kind of values like:

Version numbers:
    0.5.4
    1.0.0
    2.10.120
RGB colors:
    red
    == 255.0.0
    blue
    == 0.0.255
    orange
    == 255.150.10
or IPv4 addresses:
    127.0.0.1
    192.168.1.1
    255.255.0.0
Once a tuple! value is created, its size cannot be changed anymore (it is not a series!), but its elements can still be modified, using, for example, path syntax.
    ip: 192.168.0.0
    ip/1
    == 192
    ip/4: 1
    ip
    == 192.168.0.1
The following actions are supported by tuple! values: random, add, divide, multiply, remainder, subtract, and, or, xor, length?, pick, poke, reverse.
    1.2.3 + 100.10.1
    == 101.12.4

    255.0.0 or 0.128.0
    == 255.128.0

    192.168.2.1 and 255.255.0.0
    == 192.168.0.0

    green + blue = cyan
    == true

    red + green = yellow
    == true
Math operations are allowed with some other scalar datatypes, like integer!, float! and percent!.
    1.2.3 * 2
    == 2.4.6

    red * 33%
    == 84.0.0

    blue / 2
    == 0.0.127
Tip: in order to get the list of all predefined color names from console, just use:
    red>> help tuple!

We are considering for a future release, the option of extending the tuples with 3 or 4 elements to 16-bit values, as the internal storage space allow us to do so. This would make possible to support color values with up to 16-bit per component and version numbers with high build values.

Map! datatype

This datatype provides a dictionary-like data structure, to make it easy to store key/value pairs while providing very fast lookups. Internally, keys are stored in an hashtable, using our existing MurmurHash3 implementation.

The map! datatype has its own literal format:
    #(a: 3 b: "hello" c: 10)
    == #(
        a: 3
        b: "hello"
        c: 10
    )
Keys can be any value among the following datatypes: any-word!, any-string!, integer!, float!, char!.

Reading and setting keys and values in maps can be done using the familiar path syntax.
    m: #(a: 3 b: "hello" c: 10)
    m/a
    == 3
    m/b: 14
    m
    == #(
        a: 3
        b: 14
        c: 10
    )
In addition to that, some actions can be used when keys are not words, or when paths are not the best fit.
    select m 'a
    == 3
    put m 'a 42
    put m "Monday" "pizza"
    m
    == #(
        a: 42
        b: 14
        c: 10
        "Monday" "pizza"
    )
The new put action works as a counterpart of select, changing the value associated with a key. It will be extended to act on all series in the future.

Read more about map! in the specification document.

Exception handling

Red now provides a more complete exception system, allowing to throw and catch exceptions in a convenient way.

Syntax
    throw <value>
    throw/name <value> <name>

    catch [...]
    catch/name [...] <names>

    <value> : any value
    <name>  : a word for naming the exception
    <names> : a name or block of names
A throw will interrupt the code flow and go back up through the call stack, until a catch is reached, then resume execution just after it. By default, exceptions are anonymous, but for finer-grained control, they can be named through the /name refinement, targeting one or several specific catch statements.
    catch  [print "Hello" throw 1 print "John"]
    "Hello"
    == 1
Catch will catch all thrown exceptions and return the thrown value. If /name is used, only the matching named exceptions will be caught, all the others will pass through it.

If an exception is not caught by a catch call, it will result in a runtime error.
    red>> throw 1
    *** Throw error: no catch for throw: 1
    *** Where: throw
In order to improve the ability to handle both errors and thrown exceptions at the same time, try has been extended with a new /all refinement, allowing it to catch all possible forms of exceptions, including return, exit, break and continue misuses. It is an ultimate barrier in your code, to handle runtime issues.

Set operations

Set operations are now fully supported:
  • union: returns the union of two data sets.
  • exclude: returns the first data set less the second data set.
  • intersect: returns the intersection of two data sets.
  • difference: returns all the values which differ from two data sets.

Those operations can be applied on following datatypes: block!, string!, bitset!, typeset!. (Hash! datatype support will be added in the next release)

In order to produce sets out of block! and string! values, a unique operation is provided:
  • unique: returns the data set with duplicates removed.
    union [a b c] [d a hello 123]
    == [a b c d hello 123]

    intersect [2 6 3 4 2] [5 6 9 4 3]
    == [6 3 4]

    unique "hello red world"
    == "helo rdw"
A /case refinement is provided for performing case-sensitive set operations (default is case-insensitive).

The /skip refinement allows to process series arguments in a record-oriented way.

Note: all these operations are implemented using hashtables internally, in order to ensure the best performances.

New natives

as-pair

Returns a pair! value made out of two integer arguments.
    as-pair <x> <y>

    <x> : integer x value
    <y> : integer y value
Example:
    as-pair 2 3
    == 2x3
break

Exits a loop and resume evaluation after it. Can optionally return a value from the loop.
    break
    break/return <value>

    <value> : returned value, any type.
Example:
    loop 3 [print "hi!" break print "hidden"]
    hi!
continue

Interrupts a loop evaluation flow and resume at next loop iteration.
    continue
Example:
    loop 3 [print "hi!" continue print "hidden"]
    hi!hi!hi!
extend

Adds pairs of keys and values to an aggregate value. If the key is already defined, it will replace its value. Additions are done in a case-preserving way. Default lookups are case-insensitive, a /case refinement can optionally enforce case-sensitivity.
    extend <aggregate> <spec>
    extend/case <aggredate> <spec>

    <aggregate> : an object! or a map! value
    <spec>      : a block of key and values pairs
Note: support for object! is not implemented yet.

Example:
    m: #(a: 3)
    extend m [b: 4 "c" 123]
    m
    == #(
        a: 3
        b: 4
        "c" 123
    )
New action

A new put action has been added in order to provide a modifying counterpart to the select action. Select offers a way to access series values as if they were associative data structures. Put and select are natural accessor actions for objects and maps. Default lookups are case-insensitive.
    put <aggregate> key value
    put/case <aggregate> key value
Note: PUT support is only implemented for map! values for now, series and objects support will come in a future release.

Example:
    m: #(a: 1)
    put m 'a 2
    m
    == #(
        a: 2
    )
New function

cause-error

Causes an immediate error throw, with the provided information. Use it to generate standard or custom errors.
    cause-error <type> <id> <arguments>

    <type>      : word representing an error class
    <id>        : word representing an error definition
    <arguments> : a block of optional arguments (can be empty)
Error classes and definitions can be inspected using:
    help system/catalog/errors
Example:
    red>> cause-error 'math 'zero-divide []
    *** Math error: attempt to divide by zero
    *** Where: do
Red/System additions

New natives were added in order to better support new Red features:


Other changes

The global symbol table is now hashed, so lookups are now very fast and the startup time of Red runtime greatly improved. On fast modern hardware, there is no noticeable difference, but on low-end or old hardware, it can be very significant.

Set and get natives have been extended to allow path arguments, to access a word within an associative structure (an object or a map). Moreover, they also feature a /case refinement now, allowing to distinguish target words by case (useful for maps).

Collation tables that were visible in system/locale are temporarily hidden now, until we figure out a better way to expose that feature to users, without allowing nasty errors caused by race conditions (loading a script that changes the tables unexpectedly, concurrency issues,...)

More changes:
  • exit/return are now defined as natives instead of volatile keywords.
  • do can accept error! values.
  • parse and load are now more stable when errors are raised from parsing rules.
  • load errors handling greatly improved (no console exit on syntax errors anymore).
  • value? now supports any type, except unset! as argument.
  • fixed bugs and little improvement of help output.
  • minor Redbin speed and generated payload size improvement.
  • prin output in console fixed.
  • fixed Red/System's #get directive not working in some cases.
  • system/words now defined as an object!.
  • compiler now supports system/words/ prefix to access global context words.
  • many fixes and improvements on vector! datatype, especially on math operations.
  • color definitions are now available.
  • vector! unit tests significantly extended.
  • an op! used without arguments in the interpreter now reports an error.
  • pick and poke now accept a logic! value as index.
  • added missing comparison operators for vector!.
  • paths evaluation errors in interpreter are now more accurate.
  • first memory frame allocation increased from 512KB to 1MB.
  • fixed memory corruptions caused by function with refinements in interpreter.
  • division by zero now properly caught for floats.
  • last but not least, 44 bugs reported on Github's tracker fixed in this release!

Also, thanks to PeterWAWood for the huge work on bringing a big number of new unit tests (especially for vector! datatype).

Pre-compiled runtime

This was originally the main feature planned for the release: Compile the runtime library once and cache it on disk to speed-up compilation. Unfortunately, it has been more challenging than we expected, because the Red/System compiler constructs many word values that cannot be properly serialized by Rebol's mold action, so cannot be load-ed back. Writing ad-hoc serializers and loaders was taking too much time, so this feature was postponed for a future release. We will probably implement shadow objects first, in the Red/System compiler, and reduce the use of non-serializable words before supporting the pre-compiled runtime. It is still a high priority, because it is the first step towards modular compilation.

Moving to Gitter

We moved our chat rooms to Gitter. It's not perfect, but nicley combines the features of Stackoverflow chat and AltME, where the Rebol and Red community meet. So far, so good. Gitter is still young, but very promising. You just need a Github account, which is a lower entrance barrier than Stackoverflow chat and its required 20 reputation points. Come there, say Hello, and ask questions about Red. :-)

What's next?

This release marks the beginning of the re-integration of the android branch in master. That branch has grown up a lot in the last 12 months, to the point where merging back changes from master has become a long, complex and error-prone process. The pair! and tuple! datatype implementations were pulled from the android branch, and more code will be pulled in the next release in order to close the gap between the two branches.

Finally, the time has come to have official GUI support, and that is what the next release (0.6.0) will bring. Trying to build the GUI engine, the GUI DSL, and the Android back-end at the same time was not the best plan. The development cycle on Android is slow and debugging options limited. So Windows will be the first GUI target, where we can quickly complete the engine and DSL. Then we will merge the Android GUI back-end and toolchain in a 0.6.1 release. Those two releases should come quickly, so don't go on holiday for too long this summer. ;-)

In the meantime, enjoy this new release! :-)

The Red Team.

April 26, 2015

0.5.3: Faster compilation and extended vector! support

The main point of this minor release is to speed up compilation time by introducing a new way for the compiler to store Red values required for constructing the environment during the runtime library startup.

Introducing Redbin

Red already provides two text-oriented serialization formats, following the base Rebol principles. Here are the available serialization formats now in Red with some pros/cons:
  • MOLD format
    • provides a default readable text format, very close to the source code version
    • cannot properly encode many values
  • MOLD/ALL format
    • can encode series offsets
    • some values with literal forms that rely on words that can be natively encoded (none, true/false, objects, ...)
    • human-readable, but not always nice-looking
  • Redbin format
    • can encode any value accurately
    • supports words binding
    • can encode contexts efficiently
    • supports cycles in blocks
    • can encoded name/value pairs in any context
    • extremely fast loading time
    • very small storage space used when compressed
    • non human-readable
So far, the existing environment source code (mostly block values) was converted to pure Red/System construction code which was pretty simple and straightforward to implement, but was generating thousands of extra lines of code, slowing down the native compilation process. The right solution for that was to introduce a new binary serialization format for Red values called Redbin (very inspired by Carl's REBin proposal).

Redbin's specification focuses on optimizing the loading time of encoded values, by making their stored representation very close to their memory representation, bypassing the parsing and validation stages. Moreover, the Redbin payload is compressed using the Crush algorithm (that Qtxie ported to Red/System), which features one of the fastest decompressors around while having a general compression ratio very close to the deflate algorithm (but compression speed is about an order of magnitude slower). This fits perfectly the needs for our Redbin use-case.

So the gains compared to pre-0.5.3 version are:
  • compilation time of empty Red program is ~40% faster!
  • generated executable of empty Red program is about 100KB smaller (278KB only on Windows now).
  • faster startup time, as the Redbin decoding process is much faster than the previous Red-stack-oriented construction approach.
Those benefits also extend to user code, your static series will be saved in Redbin format as well.

Redbin format is currently emitted by the compiler and decoded by the Red runtime, but there is no encoder yet in the runtime that would allow user code to emit Redbin format. We will provide that support in a future version, it is not high priority for now. A "compact" version of the encoding format will also be added, so that Redbin can also be a good choice for remote data exchange.

Compilation from Rebol console

For those using Red toolchain from Rebol2 console, a new rc function is introduced to avoid reloading the toolchain on each run. Typical session looks like this:

    >> do %red.r
    >> rc "-c tests\demo.red"

    -=== Red Compiler 0.5.3 ===-

    Compiling /C/Dev/Red/tests/demo.red ...
    ...compilation time : 416 ms

    Compiling to native code...
    Script: "Red/System PE/COFF format emitter" (none)
    ...compilation time : 12022 ms
    ...linking time     : 646 ms
    ...output file size : 284160 bytes
    ...output file      : C:\Dev\Red\demo.exe
 
    >> call/output "demo.exe" s: make string! 10'000
    == 0
    >> print s

      RedRed              d
      d     d             e
      e     e             R
      R     R   edR    dR d
      d     d  d   R  R  Re
      edRedR   e   d  d   R
      R   e    RedR   e   d
      d    e   d      R   e
      e    R   e   d  d  dR
      R     R   edR    dR d

Collation tables

Since 0.5.2, Red provides collation tables for more accurate case folding support. Those tables can now be accessed by users using these paths:
    system/locale/collation/upper-to-lower
    system/locale/collation/lower-to-upper
Each of these tables is a vector of char! values which can be freely modified and extended by users in order to cope with some specific local rules for case folding. For example, in French language, the uppercase of letter é can be E or É. There is a divide among French people about which one should be used and in some cases, it can just be a typographical constraint. By default, Red will uppercase é as É, but this can be easily changed if required, here is how:

    uppercase "éléphant"
    == "ÉLÉPHANT"
    
    table: system/locale/collation/lower-to-upper
    foreach [lower upper] "àAéEèEêEôOûUùUîIçC" [table/:lower: upper]

    uppercase "éléphant"
    == "ELEPHANT"

Extended Vector! datatype

Vector! datatype now supports more actions and can store more datatypes with different bit-sizes. For integer! and char! values, you can store them as 8, 16 or 32 bits values. For float!, it is 32 or 64 bits. Several syntactic forms are accepted for creating a vector:
    make vector! <slots>
    make vector! [<values>]
    make vector! [<type> <bit-size> [<values>]]
    make vector! [<type> <bit-size> <slots> [<values>]]

    <slots>    : number of slots to preallocate (32-bit slots by default)
    <values>   : sequence of values of same datatype
    <type>     : name of accepted datatype: integer! | char! | float!
    <bit-size> : 8 | 16 | 32 for integer! and char!, 32 | 64 for float!
The type of the vector elements can be inferred from the provided values, so it can be omitted (unless you need to force a bit-size different from the values default one). If a value with a bit-size greater than the vector elements one, is inserted in the vector, it will be truncated to the bit-size of the vector.

For example, creating a vector that contains 1000 32-bit integer values:
    make vector! 1000
Or if you want to specify the bit-size of the vector element:
    make vector! [char! 16 1000]
    make vector! [float! 64 1000]
You can also initialize a vector from a block as below:
    make vector! [1.1 2.2 3.3 4.4]
Again you can also specify the bit-size of the vector element:
    make vector! [integer! 8 [1 2 3 4]]
For integer! and char! vectors, you can use all math and bitwise operators now.
    x: make vector! [1 2 3 4]
    y: make vector! [2 3 4 5]
    x + y
    == make vector! [3 5 7 9]
In case of different bit-sizes, the resulting vector will be using the highest bit-size. If a math operation is producing a result that does not fit the bit-size, the result is currently truncated to the bit-size (using a AND operation). Ability to read and change the bit-size of a vector will be added in future releases.

The following actions are added to vector! datatype: clear, copy, poke, remove, reverse, take, sort, find, select, add, subtract, multiply, divide, remainder, and, or, xor.

The vector! implementation is not yet final, some of its actions will get optimized for better performances and, in future, rely on SIMD for even faster operations. For multidimensional support, it will be implemented as a new matrix! datatype in the near future, inheriting from vector!, so the additional code required will be kept minimal.

Bugfixing

This was a short-term release, but we managed to fix a few bugs anyway.

What's next

Another minor release will follow with many runtime library additions and new toolchain improvements. See the planned features for 0.5.4 on our Trello board.

The 0.6.0 release will also most probably be split in two milestones, one for GUI and another for Android support.

In the meantime, enjoy this new release! :-)

April 4, 2015

0.5.2: Case folding and hash! support

This is minor release mainly motivated by the need to fix some annoying issues and regressions we have encountered in the last release:

  • the help function was displaying an error when used with no arguments, preventing newcomers from seeing the general help information
  • the console pre-compilation issue with timezones was back.

Some significant new features managed to sneak into this release too, along with some bugfixes.

Case folding

Red now provides uppercase and lowercase natives and more generally, better support for Unicode-aware case folding. Red runtime library contains now a general one-to-one mapping table for case folding that should cover most user needs.
    red>> uppercase "hello"
    == "HELLO"
    red>> uppercase/part "hello" 1
    == "Hello"
    red>> uppercase "français"
    == "FRANÇAIS"
    red>> uppercase "éléphant"
    == "ÉLÉPHANT"
    red>> lowercase "CameL"
    == "camel"
This applies also to words, so now case insensitivity is Unicode-aware in Red:
    red>> É: 123
    == 123
    red>> é
    == 123
    red>> "éléphant" = "ÉLÉPHANT"
    == true
    red>> "éléphant" == "ÉLÉPHANT"
    == false
For special cases, we will expose, in a future release, the collation table we use internally, so that anyone can provide a customized version that is a better fit for some local special rules or usages. For example, some lower case characters (such as "ß") actually map to two or more upper case code points ("SS" in this case). So in Red, by default, you will get:
    red>> lowercase "ß"
    == ß
    red>> uppercase "ß"
    == ß
You can read more about our plans for full Unicode support on the wiki.

Hash datatype

The new hash! datatype works exactly the same way as in Rebol2. It provides a block-like interface but with fast lookups for most values (block series can be stored in hash! too, but they will not be hashed, so no faster access). It is a very flexible container for any kind of hashed tables (not only associative arrays) while keeping the handy navigational abilities of blocks. The underlying hashing function is a custom implementation of the MurmurHash3 algorithm. Some usage examples:
    red>> list: make hash! [a 123 "hello" b c 789]
    == make hash! [a 123 "hello" b c 789]
    red>> list/c
    == 789
    red>> find list 'b
    == make hash! [b c 789]
    red>> dict: make hash! [a 123 b 456 c 789]
    == make hash! [a 123 b 456 c 789]
    red>> select dict 'c
    == 789
    red>> dict: make hash! [2 123 4 456 6 2 8 789]
    == make hash! [2 123 4 456 6 2 8 789]
    red>> select/skip dict 2 2
    == 123

A map! datatype (strictly associative array) should also be provided in the next release, though, we are still investigating some of its features and use-case scenarios before deciding to release it officially.

Good news also about our Mac build server, a new one was kindly provided by Will (thanks a lot for that).

Our next release should mainly feature the Redbin format support for the Red compiler, providing much faster compilation times and reduced generated binaries.

Enjoy! :-)

March 15, 2015

0.5.1: New console and errors support

This new release brings many new features, improvements and some bugfixes that will make Red more usable, especially for newcomers. The initial intent for this release was just to replace the existing console implementation, but it looked like the right time to finally implement also proper general error handling support.

New console engine

The old console code we were using so far for the Red REPL was never meant to last that long, but as usual in software development, temporary solutions tend to become more permanent than planned. Though, the old console code really needed a replacement, mainly for:
  • removing the dependency to libreadline and libhistory, they were creating too many issues on the different Unix platforms, so became troublesome for many newcomers.
  • having a finer-grained control over keystrokes on text input, in order to implement convenient features like word completion.
  • having a bigger platform-independent part, so that we can add any kind of backends, like GUI ones, without duplicating too much code.
So, the new console code gets rid of third-party libraries and runs only on what the OS provides. The new features are:
  • built-in history, accessible from system/console/history
  • customizable prompt from system/console/prompt
  • word and object path completion using TAB key
  • ESC key support for interrupting a multi-line input
Other notable console-related improvements:
  • about function now returns also the build timestamp.
  • what function has now a more readable output.
  • Console output speed on Windows is now very fast, thanks to the patch provided by Oldes for buffered output.
The console code is not in its final form yet, it needs to be even more modular and wrapped in a port! abstraction in the future.

Errors support

Red now supports first class errors as the error! datatype. They can be user-created or produced by the system. The error definitions are stored in the system/catalog/errors object.
 red>> help system/catalog/errors
 `system/catalog/errors` is an object! of value:
     throw            object!   [code type break return throw continue]
     note             object!   [code type no-load]
     syntax           object!   [code type invalid missing no-header no-rs-h...
     script           object!   [code type no-value need-value not-defined n...
     math             object!   [code type zero-divide overflow positive]
     access           object!   [code type]
     user             object!   [code type message]
     internal         object!   [code type bad-path not-here no-memory stack...

User errors can be created using make action followed by an error integer code or a block containing the category and error name:
 red>> make error! 402
 *** Math error: attempt to divide by zero
 *** Where: ???

 red>> make error! [math zero-divide]
 *** Math error: attempt to divide by zero
 *** Where: ???

These examples are displaying an error message because the error value is the returned value, we still need to implement a full exception handling mechanism using throw/catch natives in order to enable raising user errors that can interrupt the code flow. The error throwing sub-system is implemented and used by the Red runtime and interpreter, just not exposed to the user yet.

Errors can be trapped using the try native. An error! value will be returned if an error was generated and can be tested using the error? function.
 red>> a: 0 if error? err: try [1 / a][print "divide by zero"]
 divide by zero
 red>> probe err
 make error! [
    code: none
    type: 'math
    id: 'zero-divide
    arg1: none
    arg2: none
    arg3: none
    near: none
    where: '/
    stack: 3121680
 ]
 *** Math error: attempt to divide by zero
 *** Where: /
Currently the console will display errors if they are the last value. That behavior will be improved once the exception system for Red will be in place.

Errors when displayed from compiled programs, provide calling stack information to make it easier to locate the source code where the error originated from. For example:
    Red []
    
    print mold 3 / 0
will produce the following error once compiled and run:
    *** Math error: attempt to divide by zero
    *** Where: /
    *** Stack: print mold /

SORT action

Sorting data is now supported in Red, in a polymorphic way as in Rebol. The sort action is very versatile and useful. Let's start from a basic example:
    scores: [2 3 1 9 4 8]
    sort scores
    == [1 2 3 4 8 9]
As you can see, sort modifies the argument series, you can keep the series unchanged by using copy when passing it as argument:
    str: "CgBbefacdA"
    sort copy str
    == "aABbCcdefg"
    sort/case copy str
    == "ABCabcdefg"
    str
    == "CgBbefacdA"
By default, sorting is not sensitive to character cases, but you can make it sensitive with the /case refinement.

You can use /skip refinement to specify how many elements to ignore, it's handy when you need to sort records of a fixed size.
    name-ages: [
        "Larry" 45
        "Curly" 50
        "Mo" 42
    ]
    sort/skip name-ages 2
    == ["Curly" 50 "Larry" 45 "Mo" 42]
The /compare refinement can be used to specify how to perform the comparison. (It does not yet support block! as argument)
    names: [
        "Larry"
        "Curly"
        "Mo"
    ]
    sort/compare names func [a b] [a > b]
    == ["Mo" "Larry" "Curly"]
Combining it with /skip refinement, you can do some complex sorting task.
    name-ages: [
        "Larry" 45
        "Curly" 50
        "Mo" 42
    ]
    sort/skip/compare copy name-ages 2 2    ;-- sort by 2nd column
    == ["Mo" 42 "Larry" 45 "Curly" 50]
The /all refinement will force the entire record to be passed to the compare function. This is useful if you need to compare one or more fields of a record while also doing a skip operation. In the following example, sorting is done by the second column, in descending order:
    sort/skip/compare/all name-ages 2 func [a b][a/2 > b/2]
    == ["Curly" 50 "Larry" 45 "Mo" 42]
Sort uses Quicksort as its default sorting algorithm. Quicksort is very fast, but it is an unstable sorting  algorithm. If you need stable sorting, just add /stable refinement, it will then use Merge algorithm instead to perform the sort.

New datatypes

A couple of new datatypes were added in this release, mostly because of internal needs in Red runtime to support the new features.

The typeset! datatype has been fully implemented, and is on par with the Rebol3 version. A typeset! value is a set of datatypes stored in a compact array of bits (up to 96 bits). Datatype lookups are very fast in typesets and they are mostly used internally for runtime type-checking support. The following actions are supported on typeset! values: make, form, mold, and, or, xor, complement, clear, find, insert, append, length?. Comparison operators are also supported.

A preliminary implementation of the vector! datatype is also part of this release. A vector! value is a series of number values of same datatype. The internal implementation uses a more compact memory storage format than a block! would do, while, on the surface, behaving the same way as other series. Only 32-bit integer values can be stored for now in vectors. The following actions are supported by vector! values: make, form, mold, at, back, head, head?, index?, insert, append, length?, next, pick, skip, tail, tail?. The implementation will be completed in future releases.

Runtime type checking support

It has finally being implemented, as proper error handling support is now available. So from this release on, function arguments types will be check against the function specification and non-conforming cases will result in an error. Return value type-checking will be added later.

The type-checking might break some existing Red code around that was letting silently pass invalid arguments, so check your code with this new release before upgrading.

The compiler does not do any type checking yet, that will be added at a later stage (though, don't expect too much from it, unless you annotate with types every function exhaustively).

Also notice that the runtime type-checking implementation is making the Red interpreter a little bit faster, thanks to a new optimized way to handle function specification blocks (an optimized spec block is cached after first call, resulting in much faster processing time afterwards).

Red/System improvements

Exceptions handling has been improved, introducing the catch statement allowing to catch exceptions using an integer filtering value. Here is a simple example in the global context:

    Red/System []

    catch 100 [
        print "hello"
        throw 10
        print "<hidden>"
    ]
    print " world"
will output
    hello world

The integer argument for catch intercepts only exceptions with a lower value, providing a simple, but efficient filtering system.

In addition to that, uncaught exceptions are now properly reporting a runtime error instead of passing silently. This new enhanced low-level exception system is supporting the new higher-level Red error handling system.

A couple of new compiler directives have been also added in order to strengthen the interfacing with Red layer:
    #get <path>
The #get directive returns a red-value! pointer on a value referred by a Red object path. This is used internally in the runtime to conveniently access the Red system object content from Red/System code. This directive will be extended in the future to access also words from Red global context.

    #in <path> <word>
The #in directive returns a red-word! pointer to a Red word bound to the object context referred by path.

What's next?

In addition to many minor pending improvements, we will be working on a minor release that will introduce the Redbin format for accurately serialize Red values in binary form. Redbin format will be used to make the compilation process much faster, as it currently slows down pretty quickly as the Red-level environment code size grows up.

Enjoy this new release! :-)
Fork me on GitHub