July 17, 2017

0.6.3: macOS GUI backend

This new Red release contains ~830 commits, closes 86 issues, and brings a large number of  new features. The main one is official support for the macOS GUI, through a new backend for the View engine.


For readers not familiar with our GUI system, you can get a quick overview in a previous article.

In a nutshell, it is a dynamically-built reactive object tree, with:
  • pluggable backends (full support for Windows and prototype for GTK and Android)
  • full 2D vector graphics support (a large subset of SVG)
  • direct reactive programming support (not FRP style for now)
  • a declarative DSL for GUI building and layout management
  • a declarative DSL for 2D graphics

The macOS backend supports all the same features as the Windows backend, except the following, yet to be implemented:
  • fill-pen pattern, fill-pen bitmap and diamond gradient in Draw.
  • matrix transformations on pen and fill-pen in Draw.

The GUI console for macOS will be available in the 0.6.4 branch, so is not part of this release. The View module, though, is compiled in the CLI console by default on macOS, so no show-stopper there.

Here are a few short examples, starting with a GUI Hello World!:
    view [text "Hello Mac World!"]

A tiny demo script (hopefully, it should get us an office at Station F, right? ;-)):
    view [
        t: text "French Touch! " on-time [move t/text tail t/text]
        base 21x15 draw [
            pen off
            fill-pen blue  box 0x0  6x14
            fill-pen white box 7x0  14x14
            fill-pen red   box 14x0 20x14
        ] return
        button "Start" [t/rate: 10]
    ]

A small reactive text size measuring tool (one of our test scripts):
    view [
        style txt: text right
        txt "Text" f: area 200x80 
            font [name: "Comic Sans MS" size: 15 color: black]
            return

        txt "Size in pixels" text "0x0"
            react [[f/text f/font] face/text: form size-text f]
            return
 
        txt "Font name" drop-list 120
            data  ["Arial" "Consolas" "Comic Sans MS" "Times"]
            react [f/font/name: pick face/data any [face/selected 3]]
            return
  
        txt "Font size" s: field "15" react [f/font/size: s/data]
        button "+" bold 40 [s/data: s/data + 1]
        button "-" bold 40 [s/data: max 1 s/data - 1]
        return
    ]


Generating macOS bundle apps is made simple by our toolchain, just use the -t macOS target when compiling, for example using our SVG Tiger demo:
   $ red -o ~/Desktop/ -t macOS tiger.red
This will produce a tiger.app bundle on the Desktop:



Cross-Platform GUI Metrics

In order to cope with different UI guidelines across GUI platforms, this release also introduces an experimental rule-oriented GUI rewriting engine, capable of modifying a face tree dynamically according to pre-set rules. It is integrated as a last stage in VID (Visual Interface Dialect) processing. The currently implemented rules are per-platform for now (more general, cross-platform rules are also planned):

Windows rules:
  • color-backgrounds: color the background of some colorless faces to match their parent's color
  • color-tabpanel-children: Like color-backgrounds, but tab-panel specific
  • OK-Cancel: buttons ordering rule, puts Cancel/Delete/Remove buttons last
macOS rules:
  • adjust-buttons: use standard button sub-classes when buttons are narrow enough
  • capitalize: capitalize widget text according to macOS guidelines
  • Cancel-OK: buttons ordering rule, puts Ok/Save/Apply buttons last

To give you a quick grasp about why that feature matters and what it is capable of, here is a simple example, which leverages the buttons ordering and capitalization rules:
    view [
        text "Name" right 50 field return
        text "Age"  right 50 field return
        button "ok" button "cancel"
    ]

You can see, side-by-side, the macOS and Windows generated forms. Notice the button text and ordering, yes, they differ from the source code! The GUI rules have ensured that:
  • the buttons are ordered according to each platform's guidelines, "Ok" last on macOS, "Cancel" last on Windows.
  • the button's labels are properly capitalized on macOS.

This small example just demonstrates the concept, but actually, there is no limit on how much it could alter the UI and how far it could go with the post-processing.

As it is still experimental, if it causes you any trouble, you can easily disable it using:
    system/view/VID/GUI-rules/active?: no
You can also remove rules selectively, by modifying the content of the following lists:
    system/view/VID/GUI-rules/OS/Windows
    == [
        color-backgrounds
        color-tabpanel-children
        OK-Cancel
    ]

    system/view/VID/GUI-rules/OS/macOS
    == [
        adjust-buttons
        capitalize
        Cancel-OK
    ]
This allows you total control where needed, but also helps you conform to UI guidelines with no effort, saving everyone time. Not only do you not have to tweak your code for each platform when you write it, when a new platform is added, you won't have to change your code to support it. In a world where getting one detail wrong may keep you out of an app store, or prevent you from being a "certified" app, this is huge. It should also be possible to use the rule engine to write a guideline linter, where you could get a report of what rules will be applied to your VID code on each platform. By "report", we don't mean just a text listing, but it could highlight elements in the UI that were processed by rules. Having a system that helps you is great. Having a system that tells you how it helped you is even better.

There are tons of possible rules we could implement, if you have ideas for great ones, please let us know by joining us in our online chat room on Gitter or, if you prefer, by posting on our mailing-list.


Other changes in 0.6.3

Core language

The biggest addition to the core language in this release is the date! datatype, which supports all Rebol's date! datatype features and more. The additions include support for a large range of ISO 8601 date formats. About 350 unit tests have been written so far. The full documentation is available here.

Here are a few examples of accepted input formats:
    >> 1999-10-5
    == 5-Oct-1999

    >> 5-October-1999
    == 5-Oct-1999

    >> 5/9/2012/6:00
    == 5-Sep-2012/6:00:00

    >> 4/Apr/2000/6:00+8:00
    == 4-Apr-2000/6:00:00+08:00

    >> 2017-07-07T08:22:23Z
    == 7-Jul-2017/8:22:23

    >> 2017-W23-5
    == 9-Jun-2017

    >> 2017-153T105000-4:00
    == 2-Jun-2017/10:50:00-04:00
Math and other operations are fully supported:
    >> d: now
    == 14-Jul-2017/16:04:07+08:00

    >> d + 10
    == 24-Jul-2017/16:04:07+08:00

    >> d + 100
    == 22-Oct-2017/16:04:07+08:00

    >> d - 24:00
    == 13-Jul-2017/16:04:07+08:00

    >> d - 01/01/2010
    == 2751

    >> d/month: d/month - 5
    >> d
    == 14-Feb-2017/16:04:07+08:00

    >> d/yearday: random 365
    >> d
    == 1-Aug-2017/16:04:07+08:00

    >> sort [1/2/2018 1999-5-3/10:30:28 29-2-1980]
    == [29-Feb-1980 3-May-1999/10:30:28 1-Feb-2018]

    >> random 01/01/2018
    == 15-Dec-1248
Red also adds three new accessors:
  • /timezone: changes the zone and adjusts time accordingly (UTC-invariant)
  • /week: gets/sets the week of the year, starting on Sunday.
  • /isoweek: gets/sets the ISO week of the year, starting on Monday.

Conversions to/from Unix epoch time are also built-in:
    >> to-integer 1/1/1970
    == 0

    >> to-integer now
    == 1500020743

    >> to-date 1000000000
    == 9-Sep-2001/1:46:40

Other Core Additions

New functions were added to the existing simple IO API:
  • delete: deletes a file or an empty folder.
  • size?: returns the size in bytes of a file.

The do native now accepts a URL argument. For example:
   do https://raw.githubusercontent.com/red/code/master/Showcase/eve-clock.red
Run-time error reports now include a compact call stack trace:
    foo: does [1 / 0]
    bar: does [foo]

    bar
    *** Math Error: attempt to divide by zero
    *** Where: /
    *** Stack: bar foo

A set of new functions for disk-cached remote file access is now available: do-thru, exists-thru?, load-thru, read-thru, path-thru. They work in the same way as their normal counterparts, except that the retrieved file is cached locally, so on their next access, the locally cached copy will be used.

The system object has been extended with:
  • system/options/cache: points to the cache folder used by the Red toolchain.
  • system/options/thru-cache: points to the cache folder used by *-thru functions.
  • system/catalog/accessors: Path accessors available, per datatype.

The browse native can now work on macOS (it opens the default browser on a given URL).

Rudolf Meijer did a huge job, gathering currently implemented rules for math operations on mixed datatypes. This has resulted in documenting and improving those rules, better defining the resulting datatype for all math operations.

More changes:
  • Now is now fully operational and can return current date and time.
  • Wait now accepts a time! value argument
  • Improved make and to url! construction when the spec argument is a block.
  • Auto-detect older Intel CPU on Windows platform when building the console.
  • Even? and odd? now support time! values as argument.
  • R/S compiler can now output a function address map for verbosity >= 3
  • Improved min and max natives support for pair! and tuple!(now min/max is applied per dimension to pair! and tuple!, and can work with a number! as second argument).
  • -t compilation option does not force release mode anymore if target is the same OS.
  • Source file name in now displayed on syntax errors at compile-time.

LibRed

LibRed has been improved, thanks to the work of Joa-quim on a libRed binding for the Julia language, providing Julia with the full power of Red's native GUI system. Several new functions were added to libRed API:
  • redBinary(): constructs a binary! value from an external byte array.
  • redImage(): constructs an image! value from various possible external byte arrays.
  • redGetField(): gets the value of an object's word.
  • redSetField(): sets the value of an object's word.
There is more info about them in the libRed documentation.

LibRed's robustness has also been vastly improved. LibRed API functions can now catch null pointer arguments, and even invalid pointers passed to the libRed API. In these situations, libRed returns a Red error! value.

View engine

In addition to the macOS backend, a new test backend is included in this release. This backend works by simulating a GUI backend for testing purposes. It is still experimental but works fine so far. We should be able to write a whole suite of unit tests for the View engine and VID dialect now and run them on a headless Linux server (like our Travis CI). In the same way, this backend enables Red users to unit test their GUI apps with minimal effort (though more tooling on top of the current backend is welcome).

The working principle is simple: use view/no-wait to produce your GUI without starting the event loop, then form your tests using the following model:
  • Set the focus on the face where you want to simulate an event (using set-focus).
  • Trigger an event (using do-event).
  • Test if the result of the event conforms to your expectations.

Here is an extract from a short test script:
    view/no-wait [
        hello: button "Hello" [print "ok"]
        name: field
    ]

    set-focus hello
    do-event 'click

    set-focus name
    do-event/with 'key #"4"
    do-event/with 'key #"2"
    do-event/with 'key 'enter

    probe name/text
    probe name/data
Using this test backend requires setting some options in the header (see the linked script's header), and compilation in release mode.

Another notable addition is the foreach-face function, which powers our GUI rewriting rules engine. It allows easy traversal of a face tree (depth-first), selecting only the faces you need to process. The syntax is:
    foreach-face <root> [<actions>]
    foreach-face/with <root> [<actions>][<conditions>]
    
    <root>       : root face of the tree.
    <actions>    : actions to perform on matched faces.
    <conditions> : optional conditions for selecting faces.
The first version applies the actions block to all faces, while the second one (/with) will select only faces matching the conditions block (which needs to return a true value to act on the current face). Both the actions and conditions blocks have an implicit face word defined, which refers to the current face in the tree. For example:
    do %tests/vid.red
    collect [foreach-face/with win [keep face/text][find face/text #"i"]]
will return:
    == ["China" "in panel" "option 1" "option 2" "option 3" "option 4"]
Here is a more sophisticated example using foreach-face to dynamically localize labels and adjust buttons size and position accordingly, so that they fit the text nicely:

    langs:  ["English" "French" "中文"]
    labels: [
        ["Name" "Age" "Phone #" "Cancel" "Submit"]
        ["Nom" "Age" "Tél." "Abandon" "Envoyer"]
        ["名字" "年龄" "电话" "取消" "提交"]
    ]
    set-lang: function [f event][
        root: f/parent
        condition: [all [face/text face/type <> 'drop-list]]

        list: collect [foreach-face/with root [keep face/text] condition]
        forall list [append clear list/1 labels/(f/selected)/(index? list)]

        foreach-face/with root [
            pads: any [metrics?/total face 'paddings 'x 0]
            prev: face/size/x
            face/size/x: pads + first size-text face
            face/offset/x: face/offset/x + ((prev - face/size/x) / 2)
        ][face/type = 'button]
    ]
    view [
        style txt: text right 45
        drop-list data langs select 1 on-change :set-lang return
        group-box [
            txt "Name"  field return
            txt "Age"   field return
            txt "Phone" field
        ] return
        pad 15x0 button "Cancel" button "Submit"
    ]

You can also detect window resizing events, and use the current window size to resize and position faces dynamically.

It is now also possible to change the mouse cursor shape, using pre-defined values, setting a custom image to face/options/cursor, or using the cursor keyword in VID, followed by an image! value or one of the following keywords: arrow, I-beam, hand, cross.

Here is an example:

Other changes:
  • Lit-words are now accepted in flags facet blocks.
  • New Shape dialect command: close.
  • Changed default background color to white in tab-panel (Windows).
  • Added system/view/metrics (used mostly by VID, for more accurate sizing/positioning).
  • Added accelerated? option in face, when set to true, for faster/smoother face rendering (only base faces). Z-ordering is then only honored between accelerated faces.
  • When a button has the focus, pressing enter key will trigger a click event.
  • Renamed enable? facet to the more correct enabled?.
  • New function: set-focus (sets the focus on a given face, important for GUI testing).
  • New class option in face/options, allows setting a sub-class of a native control.
  • text-list now accepts a data keyword and any-string! values as input. This lets you to feed the list with values from VID. For example:
    view [
        text-list data parse
            read https://api.github.com/repos/red/red/events
            [collect [any [thru "message" 3 skip keep to ["\n" | {"}]]]]
    ]

The above script is just a single expression using two DSLs. I let you meditate on that. ;-)


VID dialect

VID has received many enhancements too. The most significant are:
  • more accurate sizing and positioning of native widgets using OS metrics
  • addition of alignment control, extending across/below layout modes.

So, it is now possible to add optional alignment keywords for the two linear layout modes:
  • across: top, middle, bottom
  • below: left, center, right
Those keywords can also follow the return command, if changing the alignment mode of the next row/column is required. The defaults are top and left. Here is a short example:
    view [
        style chk: check "Option" data yes
        style vline: base 2x60 red
        across top    vline button "Ok" text "Text" field chk return
        across middle vline button "Ok" text "Text" field chk return
        across bottom vline button "Ok" text "Text" field chk
    ]

Other changes:
  • New on-created event, triggered just after a face has been shown for the first time.
  • New strike option for faces: sets the strikethrough font style.
  • New init [<body>] styling option allows you to define a block of code to run when the style is instantiated.
  • later option added to react keyword.
  • Allows data and at keyword argument to be an arbitrary Red expression (see the above text-list example).
  • More robust face options parsing.
  • The default actor for h1 to h5 and text widgets is now on-down.
  • VID now saves the style name for each face in face/options/style.

Parse dialect

A new case command has been added to Parse dialect. Case temporarily changes the case matching mode, for the next rule only.

Syntax:
  case <mode> <rule>

  <mode>: TRUE (case-sensitive matching) or FALSE (case-insensitive) (logic!, word!)
  <rule>: rule on which the local case matching mode is applied to.

Red/System dialect

For newcomers, Red/System is Red's built-in system programming dialect, offering a Red-like syntax with C-level semantics (and many improvements over C, like namespaces, exceptions, RTTI,...).

This release brings an important improvement: structures can now be passed and returned by value on internal and external function calls. They can also now be inlined in other structures. This is achieved by putting the value keyword after a struct! type declaration:
 name [struct! [<fields>] value]
Passing structs by value is sadly not formally specified in C language, resulting in incompatible compiler-specific ABIs. Moreover, those ABIs are not documented, or only partially, so gathering all the right information was painful and time consuming. Just to illustrate how bad the situation is, after spending days researching all the various C compilers ABI, I was able to answer a stackoverflow question about returning C structs by value, which was left unanswered for 6 years... I might write a synthetic article about those C ABIs someday, from my notes, as I could not find such documentation online. Anyway, the hard work has been done for you. Red/System now implements the most common ABI for each platform:
  • Windows: Microsoft Visual C ABI
  • Linux: gcc/Clang ABI (with support for ARM-specific ABI) 
  • macOS: Clang ABI

In addition, more stack-oriented features are now supported:

The new use feature allows you to create local variables anywhere in a function body. It is useful for splitting long functions and creating local variables in macros.

The ARM backend has also been enhanced significantly in this new release:
  • fixes and improvements to soft integer division routine.
  • fixes non-passing unit tests on floats.
  • better AAPCS conformity and various bug fixes.
  • improved accuracy of overflow detection in multiplication.

Console

ANSI escaping sequences are now supported in the Red CLI console, thanks to Oldes. See this pull request for more info and a screenshot.

Other console changes:
  • improved auto-completion, now completes the longest common substring.
  • auto-detect older Intel CPU (non-SSE3) on Windows platform when building console.
  • system/console/size now provides the size of the console in columns and rows

What's Next?

The 0.6.4 version will mostly focus on the GUI console, not only bringing it to macOS, but also extending it greatly. It should be merged into the master branch in the next few days, as the new GUI console is almost finished.

The 0.6.5 version is for Android, and is already being worked on in a separate, private repo. We will merge it with the red/red public repo once it's ready, because we need control on the PR timing and the Android release. It will be a big one, unlike what you've seen so far. ;-)

In the meantime, and as usual, enjoy and have fun!
Fork me on GitHub