June 11, 2024

Text-UI View backend

Last year, qtxie worked on a toy text backend project and submitted a PR for that. After some extra additions and testing recently, it has now been merged even if it is still incomplete, it is usable enough. So, in addition to the shiny GUI backends in Red/View, now we have an old-school text-based user interfaces (TUI) backend for the View engine!

The new TUI backend has currently a subset of the GUI backends features. Here is an overview:

  • View styles: base, panel, button, check, radiofield, text, progress, rich-text, image and text-list.
  • Draw commands: textlinebox, triangle, circle, ellipse (block-based for now).
  • Rich-text supported in Draw.
  • Keyboard handling: key-down and key events (which are the same event).
  • Mouse handling: disabled by default. Use system/view/platform/mouse-event?: yes to enable it.
  • Images support Truecolor (24-bit RGB) for image rendering if the terminal supports it, otherwise it falls back to 256 colors.
  • Timers supported through /rate facet.
  • Facets supported: /offset, /size, /text, /image, /color, /data, /enabled?, /visible?, /selected, /flags, /options, /pane, /rate, /para and /draw.
  • Flags supported: password and all-over.
  • Frames drawing using squared or rounded corners (
  • Limited ANSI escape codes support in /text facet, only Colors / Graphics Mode codes.
  • Uses 256 colors for text. It should works fine on most of the terminals.
  • Works on the big-3 platforms (Linux, macOS and Windows10/11).

The pre-built CLI console binaries on our Download page now have View/VID included by default along with the TUI backend. You can use them to test and play with the TUI code examples here and in the TUI folder.

To use the TUI backend in your own compiled code, you need to add the two following options in the Red header block:

    Needs:  'View
    Config: [GUI-engine: 'terminal]

Here are a few examples, starting with a HelloWorld!:

    view [text "Hello TUI World!"]

    Hello TUI World!

When view is invoked, an event loop is launched. In order to return back to the console prompt, press the Escape key.

Here is an animated example using a progress bar:

    view [
        bar: progress 30% rate 10 on-time [
            face/data: remainder (face/data + 10%) 100%
            info/text: form face/data
            info/font/color: random white
        ]
        info: text 4 font-color white "30%"
    ]

Spinners are also fun to watch:

Here is a rich-text example. Press TAB key to switch focus between the buttons. Press Enter key to push the button.

    btn-quit: rtd-layout [i/red ["Q"] "uit"]

    view compose/deep [
        rich-text 40x3 transparent data [
            yellow "Hello"  white red " Red "  green "World!^/"
            u "Underline" /u " " s "Strike" /s i " Italic" /i
        ] return
        button "button 1"
        button 4x2 draw [text 0x0 (btn-quit)] [unview]
    ]

An example of mouse support (not all the terminals have mouse support):

    system/view/platform/mouse-event?: yes

    view [panel 80x20 [base 11x1 center "drag me😀" loose]]

Simulating old-style text interfaces:

    view [
        panel navy 40x15 draw [
            pen off fill-pen black box 5x4 36x10
            fill-pen pewter pen black box 4x3 35x9
            pen red text 15x3 "Hello Red"
        ][
            origin 5x5
            rich-text 30x1 pewter data [
                green "Welcome" yellow " to" red " Red "
                u "TUI" /u blue " World!"
            ]
            return
            pad 12x1 button 4 "<OK>" [print "Hi!"]
        ]
    ]

File and folder requesters are also available in TUI, navigation is done using TAB and arrow keys, selection using Enter key:

Images support:

    url: https://upload.wikimedia.org/wikipedia/en/e/e9/Red_Language_Tower_Logo.png
    view [image url 32x15]

Here is the same image displayed in TUI next to the GUI version:


Some examples of 2D vector graphics using the Draw dialect (currently using only block graphics, braille-based graphics in the future):

    view [
        base 80x40 transparent draw [
            pen orange
            triangle 3x2 18x5 5x15
            fill-pen blue
            circle 30x8 5
            pen off
            fill-pen green
            ellipse 50x2 15x15
            pen brick fill-pen brick
            box 3x20 15x30
            pen gold
            line 20x30 28x20 40x28 44x24
        ]
    ]

A special mention to group-box widget. It has a couple of new options for the frame style:

    border-corners: round | #(none)
    border-color: #(tuple!)
Here is a usage example:
    view [
        group-box font-color green " Folders " 26x8 options [
            border-corners: round
            border-color: 255.0.0
        ]
        group-box " Files " 26x8
    ]


This TUI implementation is still not on par with our GUI backends. If some of you are motivated to extend and improve it, contributions are welcome! For example, we did not yet implement menus support. If someone is up to the task, please follow the GUI View menu dialect.

What's next?

We are finishing the work on some significant improvements to the Red and R/S memory management sub-systems and garbage collector that will bring them to the level required for a Red v1.0. Those changes will be released in a bumped 0.6.6 version. Those memory improvements are also needed for completing the work on the async IO branch.

Another version bump will follow with the deprecation of the high-level Red compiler and the addition of a new powerful layer to our Red tower of languages. All those changes are pre-requirements to start our work on 64-bit support.

In the meantime, enjoy this new toy!



May 27, 2024

Red in the real world

We're often asked what Red can be used for, or what apps have been written in Red. Red can be used to write almost anything, but the sparse ecosystem and some missing pieces limit certain use cases. It's used a lot for in-house data processing, custom DSLs, simple GUI apps, and more. We also used it to build Redlake's DiaGrammar product.

When we heard that someone had written a commercial app in Red, we thought that was great news, and we're here to tell you a little about it. Your first question is likely "What is it?" and the second "Where can I get it?". It's an XML processor, and you can find it here. The video on their site goes into detail about use cases and features, so we won't cover that here.


We asked the author to talk about why they wrote SmartXML why those chose Red for the implementation. Here's what they had to say:


Once I encountered the need to parse multiple XML files. I always thought that parsing tasks were very simple and that I wouldn't encounter any difficulties because there are things like XPath and XSD that, as I was told, solve all possible problems. However, I quickly realized that this was not the case, and some tools/standards only complicate life and are of little use for real-world usage. Thus, my XML parser project was born, which would allow working with real data rather than synthetic examples where XPath and XSD are truly effective.

I chose Red because I was tired of the complexity of 90% of modern languages and the constant breaking changes in many of them. If you were to ask me what language I would choose to start a project with, looking back, I would still choose Red or perhaps try to use Hare (even considering that it's not yet completed) simply because I want to be sure that my solution will work in 10 or even 20 years. Initially, I thought I could finish within half a year, but the project took me many years. Nevertheless, I brought the project to completion.

The main idea behind SmartXML was:

1. To make the parsing process as visual and simple as possible.
2. To abandon the use of XSD schemas, which create more problems than they solve.
3. To rethink XPath by creating a replacement that could work with complex structures.
4. To implement the ability to generate SQL from XML.
5. To implement batch processing of files.


And their advice on development in general:


During the writing process, I had a lot of time to think about what constitutes good code and a good product. Here are some of my thoughts that I realized while working on SmartXML:

1. Standard tools have standard problems. And people very often become hostages of such solutions. Most people prefer the shortest, not the most correct path.

2. The time spent on design is directly proportional to the lifetime of the software product. If you spend 10 years thinking about a problem from different angles, you are more likely to come up with an architecturally beautiful and cohesive solution, but maybe not on the first try. The easiest way to make your application 100-200 times heavier than necessary is not to try to think ahead, but to solve problems as they come.

3. You need to have the strength to admit mistakes. Even if they are design mistakes that require rewriting everything from scratch. Sometimes it is wiser to throw everything away and start over than to continue day by day moving into a dead end.

4. Sometimes it's better to do reengineering of the old instead of inventing something new.

5. A language that allows you to write code quickly solves tactical, not strategic tasks. A huge amount of code written in C 30 years ago will still be relevant in another 30 years. Rebol was designed for 20 years, so most of the code on it will still work with minimal modifications in 30 years.

6. Fighting complexity should take as much time as optimizing algorithms. Simple things are always obvious in hindsight. Writing complex code is always easier than writing simple code. Simple code will always be easier to extend and maintain, and it will always have fewer errors.

7. If you can sacrifice 10% of functionality at the cost of removing 90% of code, you should do it.


This application is a great fit for Red, whose `parse` function makes processing data easy (as much as XML processing can be) and clearly defined. The latter aspect is important, and often ignored. Can you write code to get that job done, maybe with regexes in this case? Yes. But can you maintain and extend that code? This is where dialects add enormous value. In a use case like this, being able to represent the data in Red format internally, for processing, also makes your life easier.

We thank the author of SmartXML for taking the time to talk about SmartXML with us, and we're excited to see what others do with it. Tell us about your project!

Happy Reducing!

February 19, 2024

0.6.5: Changelog

Bumping up the version number was motivated by the breaking syntax change done recently. We do not offer specific builds for a given version number anymore since we provide automatic builds (with builds history) on each new master commit. Though, the changelog for new version number changes will still be provided...and this one is pretty big as it covers about 5000 commits! Hope this will help users who might have missed some changes to catch up.

613 PRs were merged, 2415 fix commits were pushed, among which 902 are closing issues tracked on Github. The most notable new features and changes are listed below with eventual links to docs or previous blog posts describing or mentioning them:

Main new features

  • New datatypes: money!, ref!, point2D!, point3D!.
  • New codecs: RedbinJSONCSV
  • New high-performance run-time lexer with instrumentation support.(blog)(doc)
  • Interpreter instrumentation support (debugger, tracer, profiler).(doc)
  • New powerful APPLY native, with deep interpreter support.(blog)
  • Dynamic refinements support.(blog)
  • Adds compress and uncompress natives with gzib, zlib and deflate algorithms support.
  • Adds gpio:// port with GPIO dialect for RaspberryPi.(blog)
  • Adds TAB navigation support to View. (blog)
  • Adds raw strings syntax support.(doc)
  • Swaps map! and construction syntax. (blog)
  • Hashtables are now used for fast lookups in contexts.
  • Custom dtoa library implementation to load and form float values.
  • Standard library and garbage collector stability vastly improved.

Finished or almost finished features in branches::
  • Full IO ports with async support, including new IPv6! datatype.(branch)
  • TextUI backend to View.(PR)
  • XML codec.(PR)

Other general new features or changes

  • New natives: TRANSCODE, SCAN, AS-MONEY, ENHEX.
  • New functions: SINGLE?, LAST?, DT, TRANSCODE-TRACE, TRACE, CLOCK, NO-REACT, DO-NO-SYNC
  • New routines: SET-TRACE, TRACING?
  • Extends EMPTY? to support map! values.
  • Allows NONE as value in map!.
  • Adds REMOVE/KEY support for removing map! entries.
  • Adds FOREACH support for map!.
  • Add new-lines automatically when converting maps to blocks.
  • Supports issue!, money! and refinement! as key in map
  • Allows any-string! series to be matched against binary input in PARSE.
  • Adds Accept-Charset and User-agent to HTTP header.
  • Update libcrypto version requirement on Linux platforms.
  • Adds support for native port actors.
  • Extends ROUND/TO to support time!.
  • Adds a cross-platform threading library to Red runtime library.
  • Adds a FIFO MPMC queue to Red runtime library.
  • Adds history support to Red console.
  • Extends GET native to accept any-word! as argument.
  • FIND is now returning FALSE instead of NONE when used on bitsets.
  • Deprecates FIND on objects (redundant with IN).
  • Adds ABOUT function to console.
  • Preprocessor: fetch-next now supports set-words/paths, get/lit-args, object/series paths.
  • COMPLEMENT is now allowed on tuples.
  • Adds RANDOM/SECURE.
  • SORT support of /skip & /all & /compare integer extended to strings, vectors and binaries.
  • Extends FIND/LAST to support hash!.
  • Allows FINDing by datatype and typeset on hash! series.
  • Removes percent-encoding from files, use double-quotes when needed instead.
  • Adds SYSTEM/CATALOG/ACCESSORS.
  • Allows pair! for COPY/PART argument on series.
  • Allows LOOP and REPEAT to take a float argument as counter.
  • Extends REDUCE/INTO to support hash! destination.
  • Extends BODY-OF to support action! and native! values.
  • REPLACE reimplemented.
  • Adds support for REVERSE/SKIP.
  • Extends VALUE? native supports any-word! argument.
  • Enable image! in red/core on Linux.
  • Adds GC support to libRed.
  • Extends DISTANCE? to support pair! arguments.
  • Adds /KEEP refinement to TRY.
  • Implements RENAME action for FILE! datatype.
  • Adds routine arguments type-checking support to compiler.
  • Accelerates the output speed of LIST-DIR.
  • Allows error! values to be modified by users.
  • Adds [trace] and [no-trace] function attributs.
  • Faster/simpler EMPTY? function implementation.
  • Forces the inclusion of Git information in the runtime.
  • Optimization for appending values into hashs.
  • Extends CHANGE to support image! argument.
  • PICK on pair!, date! and time! values now support named accessors as index argument.
  • Adds PICK action to event! datatype.
  • Makes RECYCLE returns the allocated memory total by default.
  • Cleaner implementation of deep reactive paths support.
  • Internalizes SYSTEM/STATE/NEAR value.
  • Compound scalar datatypes (pair!date!time! and tuple!) will now emit ON-CHANGE events when one of their component is changed using an access path (both in compiled and interpreted code).
  • Adds reactivity support to bitset! values.
  • Adds support for REFLECT action on bitset! values.
  • Reports a proper path in compiled path errors.
  • Adds memory usage before/after a GC cycle in debug output.
  • Measures GC time in debug mode.
  • [ARM] Adds "division by zero" and "division overflow" checks in debug mode.

Parse

  • KEEP PICK on paren expressions now merges list of values to collected block.
  • Optimizes Parse's KEEP memory usage on strings/binaries.
  • Removes the end checking in iteration rules.
  • Speed optimizations for `TO <token>` rules.
  • Adds a fast path for `TO/THRU end` rules.
  • New set of optimizations for looping on a char! value in PARSE.

VID

  • Smarter merging of actors in a style with custom actors in the instance.
  • Added password flag for hidden input
  • Adds scrollable flag.
  • defines VID origin and spacing values per backend.
  • adds next and prev as default options to VID-made faces.

View

  • Adds a calendar widget.
  • Adds support for tri-state checkboxes.
  • Significant GTK backend improvements and upgrades to match other backends.
  • Better handling of DPI changes.
  • Handles pause and scroll-lock keys.
  • Now EVENT/PICKED is a float wheel delta-value in on-wheel event.
  • Improves user experience when closing window that contains a large number of faces.
  • Scale the font size with Ctrl + mouse wheel in GUI console.
  • Adds stop-events function to easily exit a View events loop.
  • Adds resize-ns and resize-ew mouse cursors.
  • Adds bounds option to /options facet for restricted dragging area.
  • Adds new /sync refinement to VIEW function. 
  • Adds /color facet info to DUMP-FACE output.
  • Improves memory usage when changing /draw facet content.
  • Adds support for semi-transparent no-border top-level windows (Windows).
  • Minimal dark mode support on GUI console.

Draw

  • Switch command parameters to point2D! and float! for subpixel precision.(blog)
  • New Direct2D backend for Windows.
  • Add line-pattern command for drawing dashed lines.
  • Extends scale command to support percent! values.
  • Supports image mapping to arbitrary quadrilateral.
  • Removes matrix-order command in DRAW.

Red/System

  • Adds subroutines to R/S functions.(blog)
  • Implements system/io/* instrinsics for CPU I/O read/write instructions.
  • Adds system/stack/push-all and system/stack/pop-all intrinsics.(blog)
  • Support for atomic operations using system/atomic/* intrinsics.(blog)
  • Adds #inline directive to R/S for including assembled binary code.
  • Implements support for integer hardware division (ARMv7+).
  • Generates optimized code for divisions by a power of 2 literal (ARM).
  • Vastly improved loop counter handling robustness.
  • Full support for special float values (-0.0, 1.#NaN, 1.#INF, -1.#INF).
  • Drops support for % and // operators on float types.
  • Adds support for function pointers in literal arrays.
  • Switches to 16-bytes stack alignment on Linux.
  • Adds log-2 function to standard library.
  • Allows cross-referenced aliased fields in structs defined in same context.
  • Support multiple variable assignments.
  • Allows grouping arguments and local variables type specification.
  • Relax lexical format of hexadecimal literals.
  • Allows get-paths pointers on function nested in contexts.
  • Adds support for simple pointer to pointer type.
  • Allows function! type to be specified for local variables.

Toolchain

  • Optimizes critical section in linker, twice faster linking time now on average.
  • Adds --show-func-map compilation option.
  • Various minor improvements to PE format support.
  • Adds working Linux-musl target.
  • Expands libRedRT to support View backend.
  • Adds --no-view option for Red binaries.
  • Shows the global words used by the toolchain in the compilation report.
  • Adds new Pico compilation target.

February 11, 2024

Important Change! Switching map and construction syntax.

Sometimes deep changes take a huge amount of code. Sometimes they take a lot of detailed explanation and consideration, leading to long discussions and people taking sides. Rarely does an important syntactic change to a language happen quickly, with universal agreement, simple implementation, and tools to help update scripts in the wild. Today is one of those rare days.

Admittedly, this idea has been discussed for a long time. It would surface, people nodded their virtual heads, and it would submerge again. Today it's ready to deploy. Not only that, but Rebol3 is making the same change, so the two languages will still be compatible in this regard.

What is the change?

It's easy to describe. Today, map! values use this syntax: #(...) and construction syntax (sometimes called serialized form or loadable form) looks like this: #[...]. Going forward, those syntactic forms will be swapped. Why? The answer is easy. In Redbol langs, blocks do not evaluate by default, you have to do or reduce them. Parens, on the other hand, do evaluate by default. Today, maps use paren-like syntax, but they do not evaluate, while construction syntax uses block-like syntax, but does evaluate. This is a carryover from Rebol, so the major concession here is that Red and Rebol3 will no longer be compatible with Rebol2's construction syntax.

If you've never heard of construction syntax, there's a nice explanation of it here. Red only supports a few values via construction syntax today, all datatype literals, true, false, none, and unset; but eventually it will support much more. If you look at the help for mold, you'll see that /all is TBD (very partially implemented for now), and that's how you create loadable, serialized, data that can safely and easily contain any value (like redbin but readable by humans). It helps avoid cases where none or true/false may load as words. This is also why construct evaluates those specific words (including also on/off/yes/no), but not others. When loading untrusted data, we have to strike a balance between ease of use and safety.

What do I have to do?

Not much. There are two tools available, which will convert your scripts automatically. The first is small and simple, showing just how powerful Red is, and leveraging its lexer instrumentation. You can find it here (once merged, that branch may go away and the tool will be in the main branch). The second is a more advanced and standalone tool written by @hiiamboris. You can find that here.

For the simple script it's necessary that you run it under a current version of Red's lexer. Once the change is in place, running it under the new lexer will make the exact opposite change. Of course, you can compile it into a standalone EXE, or use Boris' app, which is already available.

To run the simple script from a Red console, you can just:

  do https://raw.githubusercontent.com/red/red/master/utils/migration/map-conv.red

then you can use help map-conv to see all the available options. By default it runs in preview mode, making no changes, and showing you all the instances it found, which will be changed if you use the /save refinement. A copy of each changed file is created with a .saved extension in the same folder. If you don't want them, you can use the /no-copy refinement.

A word of warning, if you run the conversion tools a second time on the same files, they will convert the data back, because they can't know what you're thinking. On the bright side, this is an effective "undo" feature. Still, it's wise to back up your data before running any tools against them.

Thanks to the power of Red, and these tools, there are already PRs pending for updates to docs and community scripts. But the other thing you can do to help is to let us know when you find things that need to be updated for this change, and especially if you run into any issues when converting your own code.

Conclusion

We know changes like this can be hard, but better now than when there is even more code in the wild that would be affected. If we had been any other language, this long-view improvement might not have happened. Only because Red (and Redbol langs in general) can consume their own code as data and have powerful parse and lexing features, was this change so easy and safe. It's still a code-porting process, and if you run multiple versions of Red you may need to maintain separate versions for a while. The other hard part is retraining your hands and eyes to the new syntax.

Happy Reducing!

November 22, 2023

Tab Navigation

We finally got tab navigation implemented! You might think it should have been an easy feature to add, but achieving a consistent and controllable behavior across our different native GUI backends is not that straightforward. So we opted for a mixed implementation with a general high-level navigation layer in Red and left spatial navigation handling to each backend, in order to preserve the native behavior as much as possible.

Automatic navigation

By default, pressing TAB key will allow you to navigate to all the GUI widgets in a window, capable of acquiring the focus. Once the last widget is reached, the next TAB press will circle back to the first focusable widget. Conversely, back-navigation can be achieved using Shift-TAB key combination, circling from first face to last one. Here is a simple example:

    view [
        text  "Name"     field focus return
        text  "Surname"  field return
        below
        check "Single"
        check "Employed"
        button "Send"
    ]

Note: check-boxes selection/unselection is done using the Space key (default on Windows).


It is possible to make a face "TAB-transparent", so that TAB navigation will skip it in both directions. This is achieved by removing the focusable flag from a navigable face. For example, in the following code, clicking on the "Click me!" button will toggle the button's focusable flag on and off (using set-flag/toggle):

    view [
        text "Name"     field focus return
        text "Surname"  field return
        below 
        check "Single"     
        check "Employed"   
        button "Send"
        button "Click me!" 100 [
            face/text: pick ["TAB ignore" "TAB stop"] to-logic face/flags          
            set-flag/toggle face 'focusable
        ]
    ]


In case of area face, the default behavior for TAB navigation means that tab characters cannot be input in the area. In such cases, the alternative Ctrl-TAB key combination can be used to input tab characters. In case the focusable flag is removed from an area face, then TAB key will directly produce tab characters. Here is an example:

    view [
        text "Name"     field focus return
        text "Surname"  field return
        below 
       	text "Comments" com: area 
        button "Send"
        button "Toggle Area" [set-flag/toggle com 'focusable]
    ]

Note: when the focusable flag is on, Ctrl-TAB is used to input tab characters, when it's off, it's just using TAB key.


Manual override

In some cases, the user can decide to set a different path for keyboard navigation. For each navigable face (the ones with a focusable  flag), it is possible to manually define the next and/or previous one when tabbing forth and/or back. In order to do so, next and prev options can be set to define how tabbing will navigate to the next or previous face.

Here is a simple example where the default navigation is changed to jump into fields marked as invalid or empty (using pink background) after a typical form submission:

    view [
        group-box 2 [
            style error: field pink
            text "Name"     field "John"
            text "Surname"  field "Smith"
            text "Age"      error "abc" focus
            text "Address"  error "-"
            text "Zip code" field "12345"
            text "City"     error
            text "Country"  error
        ] return
        btn-send: button "Send"
        do [
            list: collect [foreach-face self [if face/color = pink [keep face]]]
            forall list [list/1/options/next: list/2]
            btn-send/options/next: list/1
        ]
    ]

Notes:

  • For the sake of simplicity in this example, only forward navigation is restricted, backward navigation will visit all focusable faces.
  • In the do block, self refers to the window face, as do denotes a global section, not widget-related.
  • When list/1 refers to the last element, list/2 returns none, so it does not point to any specific face. In such case, the default tab navigation will automatically (and conveniently) select the next face, which is the "Send" button.
  • The last line is there to connect that last face (the button) to the first face in our restricted list.


Other notable changes

An important change concerns the insert-event-func function specification, it now requires a name as argument:

    >> ? insert-event-func
    USAGE:
         INSERT-EVENT-FUNC name fun

    DESCRIPTION: 
         Adds a function to monitor global events. Returns the function. 
         INSERT-EVENT-FUNC is a function! value.

    ARGUMENTS:
         name         [word!] 
         fun          [block! function!] "A function or a function body block."


The name is an arbitrary word that only needs to be unique, so it becomes easier to check if a given global handler has been installed or not. It also makes it easier to remove it, as it can be referred by name in remove-event-func. Existing handler names can be checked using:

    >> extract system/view/handlers 2
    [tab field-sync reactors radio enter debug dragging]

Please update your code if you have been using those functions.

Enjoy!

August 9, 2023

Subpixel GUI

Maybe you didn't notice, but Red/View, our GUI engine, has subpixel precision from the beginning! Unfortunately, that level of precision was not directly accessible to end users, until now. 

Actually, it would be more accurate to say that we had subpixel resolution only so far. The guilty part is the pair! datatype being limited to integer components only, while subpixel precison requires decimal numbers. So we have recently introduced new datatypes to cope with that.

What urged us to make those changes now was a very peculiar visual glitch caused by that dissonance. That glitch happens during face dragging operations. Here is an example using our View test script:

As you can see, on some positions, the face starts shaking while the mouse cursor remains still. This affects any type of face. The shaking is about ±2 pixels. It is caused by the difference in precision between the /offset facet expressed in integer numbers and the backend API, which only deals with floating point numbers. The accumulated error when converting integer->float->integer gives a 2 pixels difference. Such error happens on displays where the scaling factor is different from 100%. With the rise of 2K, 3K and 4K displays, a scaling factor > 100% has become the norm, making this glitch more frequent. You might think that this is not a big issue until you start building custom scrollbars and see your entire scrolled content shaking massively...

New point datatypes

In order to provide decimal positions and sizes for View faces, extending the existing pair! datatype was considered, though, the pair syntax can hardly scale up for such needs:

    2343.122x54239.44
    2343.122x54239.44x6309.332
    2343.122x54239.44x6309.332x442.3321
    2.33487e9x54239.44
    2.33487e9x54239.44x9.83242e17
    2.33487e9x54239.44x9.83242e17x5223.112
    1.#infx1.#infx1.#inf

As you can notice there, it quickly becomes difficult to read and identify the individual components. So we opted for adding a new literal form (hence a new datatype) that matches how coordinates for two or more dimensions are commonly represented:

    (2343.122, 54239.44)
    (2343.122, 54239.44, 6309.332)
    (2343.122, 54239.44, 6309.332, 442.3321)
    (2.33487e9, 54239.44)
    (2.33487e9, 54239.44, 9.83242e17)
    (2.33487e9, 54239.44, 9.83242e17, 5223.112)
    (1.#inf, 1.#inf, 1.#inf)

Such literal forms requires the comma character to be a delimiter, so that it cannot be used anymore as a decimal separator. That was, unfortunately, a necessary decision in order to unlock such literal forms. The gains should be bigger than the loss.

So, two new datatypes have been added:

  • point2D!: a two-dimensional coordinate or size.
  • point3D!: a three-dimensional coordinate or size. 

Their canonical lexical forms are:

    (<x>, <y>)
    (<x>, <y>, <z>)

    where <x>, <y> and <z> are integer or float numbers.

Optional spaces are allowed anywhere inside the point literals on input, they will be removed on loading.

    >> (1,2)
    == (1, 2)
    >> (  1.35 ,  2.4  )
    == (1.35, 2.4)

Both for 2D and 3D points, their components are internally stored as 32-bit floating point numbers, so that their precision is limited to 7 digits. This should be far enough for their use-cases though.

When one of the components has a fractional part equal to zero, it is displayed without the .0 part for easier reading. Similarly, integers are accepted as input for any component and are internally converted to a 32-bit float.

    >> (0.3, 0.5) + (0.7, 0.5)
    == (1, 1)
    >> (2.0, 3.0)
    == (2, 3) 

Creation

Besides literal points, it is possible to create them dynamically, the same way as pairs, using make, to or one of the as-* native functions:

    >> make point2D! [2 4.5]
    == (2, 4.5)
    >> to-point2D 1x2
    == (1, 2)
    >> as-point3D 1 (3 / 2) 7 * 0.5
    == (1, 1.5, 3.5)

Accessors

Point components can be individually accessed using ordinal numbers or component names using action accessors or path syntax:

    >> pick (2, 4.5) 1
    == 2.0
    >> pick (2, 4.5) 'y
    == 4.5
    >> p: (2, 4.5)
    == (2, 4.5)
    >> p/x
    == 2.0
    >> p/y: 3.14159
    == 3.14159
    >> p
    == (2, 3.14159)

Math operations

Basic math operations are supported as well:

    >> (1, 1) + (2, 3.5)
    == (3, 4.5)
    >> (1, 1) - (2, 3.5)
    == (-1, -2.5)
    >> (2, 3) * (10, 3.5)
    == (20, 10.5)
    >> (20, 30) / (10, 3)
    == (2, 10)

Notice that mixing pairs with point2D in math expressions is allowed. The pair value will be promoted to a point2D in such case (as integers with floats):

    >> 1x1 + (2, 3.5)
    == (3, 4.5)

Other actions/natives

    >> round (2.78, 3.34)
    == (3, 3)
    >> round/down (2.78, 3.34)
    == (2, 3)
    >> random (100, 100)
    == (53, 81)
    >> zero? (0, 0)
    == true
    >> min (10, 100) (24, 35)
    == (10, 35)
    >> max 10x100 (24, 35)
    == (24, 100)    
Notice that pairs will be promoted to point2D in mixed use cases with min/max.

 

View and VID adjustments

The main changes are in face! object:

  • /offset: now only accepts point2D! values.
  • /size: accepts both pair! and point2D! values.

In VID, both pair and point2D values can be used to denote positions and sizes, so that VID is backward compatible. All previous VID code should work without any change. VID will convert all positions to point2D values. Sizes by default in VID, keep using pairs, unless a point2D is provided by the user.

All Draw commands that were accepting pairs now also accept point2D values for higher precision.

The related documentation will get updated soon to reflect those changes.

In order to illustrate the difference in using pairs and point2D positions, here is a (not so) simple animation comparison showing the subpixel positioning difference (correctness of animation in this case is privileged over simplicity of code):

    view/no-wait [
        size 800x200 space 0x0
        b1: box 2x40 red return
        b2: box 2x40 blue
    ]
    x: b1/offset/x
    until [
        do-events/no-wait                   ; processes GUI events in queue
        wait 0.1                            ; slows down the animation
        do-no-sync [                        ; switches to manual faces redrawing
            b1/offset/x: b1/offset/x + 0.1
            b2/offset/x: to-integer x: x + 0.1
            if all [b1/state b2/state][show [b1 b2]] ; redraws both faces
        ]
        any [b1/offset/x > 700 none? b1/state none? b2/state]
    ]

Here's the zoomed capture of the result (on a display with 200% scaling factor):


The red bar uses the newly enabled subpixel precision, while the blue bar simulates the old pair positioning precision (so only allowing integer positions). What you can see is that the red bar makes two smaller steps while the blue bar makes a single one, looking more "jumpy".

This means that now animations on displays with a scaling factor > 100% can be smoother as they benefit from more accurate positioning.

Note: the animation code above is far from being simple or elegant, we'll be working on improving that. The animation code could have been quite simpler by using a rate option in VID and putting the animation code in a on-time handler. Though, timer events firing (especially on Windows) are not very reliable, so unrolling a custom event loop lowers that risk when the timing is critical (like for fast game loops).


As a conclusion, here is an old-school style starfield code demo using 2D and 3D points (moving mouse left/right changes the stars speed):

Let us hear your feedback about those changes on our Gitter (now Matrix) channel.

Enjoy!

June 8, 2023

Dynamic Refinements and Function Application

It's Time to Apply Yourself to Red


Red has never had an apply function, though we knew it would come someday. In the meantime, some of us rolled our own. Gregg's was simple, neither flexible nor efficient, and just a couple lines of code. Boris made a much more capable version, but it could still only be so fast as a mezzanine. R2 had a mezz version, which suffered the same problem. All that changes now. Apply is dead! Long live Apply! It required deep work, and a lot of design effort, but we think you'll like the results, whether you're a high-level Reducer, or anxious to see how much leverage you can, um, apply, from a functional perspective. Everybody wins.

If you don't know what apply is, in terms of functional languages, take a moment and read up. If you can get through the introduction there without getting dizzy, great. If your head is spinning, feel free to stop after the first section of this article and ignore the deep dive. You still get 90% of the value for most high level use cases. Gregg got so dizzy that he fell down, but was still able to help with this article.

Function application is largely about composition. How you can combine functions in a concise way for maximum leverage and minimum code. The problem with its design in many languages is that it makes things harder to understand. Rather than concrete functions names, there is indirection and abstraction. It can be tricky to get right, especially in a flexible language like Red, while also maintaining as much safety as possible. You can drive fast, but still wear your seat belt.

Dynamic Refinements


This subtle feature is likely to see wide use, because it will reduce code and let people build more flexible functions. It's also easy to explain. Here's an example. First, how you would write it today:
repend: func [
    {Appends a reduced value to a series and returns the series head} 
    series [series!] 
    value 
    /only "Appends a block value as a block"
][
    either only [
        append/only series reduce :value
    ][
        append series reduce :value
    ]
]

With dynamic refinements, it can be done like this.

repend: func [
    {Appends a reduced value to a series and returns the series head} 
    series [series!] 
    value 
    /only "Appends a block value as a block"
][
    append/:only series reduce :value
]

In case you missed the subtlety, it's
:only being a get-word in the path. That's right, it's evaluated, rather than being treated literally, just like you use in selector paths. The value for the dynamic refinement is taken from its context, which can be a refinement in the function, or a local value. It can be any truthy value to use the refinement, and it is only retrieved not evaluated. That means you can't use a computed value directly, which makes it safer. Other than that, paths work just as they always have. If a refinement is used, fixed (literal) or dynamic, which can be mixed however you want, any arguments it requires will be fetched and used. Otherwise they are silently ignored, so you don't have to clutter your code or worry about what a dynamic path expression will consume.

repend: func [
    {Appends a reduced value to a series and returns the series head} 
    series [series!] 
    value 
    /only "Appends a block value as a block"
    /dup  "Duplicate the value"
        count [integer!]
][
    append/:only/:dup series reduce :value count
]

>> only: false  dup: false  repend/:only/:dup [] [1] 3
== [1]
>> only: true  dup: false  repend/:only/:dup [] [1] 3
== [[1]]
>> only: false  dup: true  repend/:only/:dup [] [1] 3
== [1 1 1]
>> only: true  dup: true  repend/:only/:dup [] [1] 3
== [[1] [1] [1]]

This is an incredibly exciting and powerful feature. It's a shame there isn't more to explain. ;^)

Function Application


Functional Programming has never yet become mainstream, though it has periodic rises in popularity and a devoted following in many language camps. Even Red. Yes, Red is a functional language. It's not a pure functional language, because functions can have side effects, but functions are first class values and can be passed around like any other. It lets you do things like this:
>> do-math-op: func [
    fn    [any-function!]
    arg-1 [number!]
    arg-2 [number!]
][
    fn arg-1 arg-2
]

== func [fn [any-function!] arg-1 [number!] arg-2 [number!]][fn arg-1 arg-2]
>> do-math-op :add 1 2
== 3
>> do-math-op :subtract 1 2
== -1

That's called a Higher Order Function, or HOF. It also means you can return a function as a result.

>> make-multiplier: func [
    arg-1 [number!]
][
    ;; We're making a function and returning it here.
    func [n [number!]] compose [(arg-1) * n]
]

== func [arg-1 [number!]][func [n [number!]] compose [(arg-1) * n]]
>> fn-m: make-multiplier 4
== func [n [number!]][4 * n]
>> fn-m 3
== 12

That's all well and good, but what if you want to call different functions that take a different number or type of arguments? Now it gets tricky, and inefficient. Because Red uses free-ranging evaluation (function args are not contained in a paren or marked as ending in any way at the call site), how do you handle different arities (number of arguments)? Here's a very simple apply mezzanine:

apply: func [
    "Apply a function to a block of arguments." 
    fn [any-function!] "Function to apply" 
    args [block!] "Arguments for function" 
    /only "Use arg values as-is, do not reduce the block"
][
    args: either only [copy args] [reduce args] 
    do head insert args :fn
]

So easy! The power of Red. But there is a cost. It's a mezzanine function, so it's slower than a native function, the args are copied or reduced, and then do is used to evaluate it. We can live with this kind of overhead for a great deal of Red code, but apply is a building block, and may be used in deep code where performance is important. You may also have noticed that the fn argument is any-function!, that means two things: 1) If you want to know the name of the function, the word that refers to it, too bad. You'd have to pass another arg for that. 2) Refinements. You can't use them with this model. And that limitation is a killer. For example, you could pass :append but not :append/:only. And there's no way you could have an /only refinement in your function and just pass that along. Until now. 

The Real Apply


Here's the new apply native that is now available in Red:

USAGE:
    APPLY func args

DESCRIPTION: 
    Apply a function to a reduced block of arguments. 
    APPLY is a native! value.

ARGUMENTS:
    func    [word! path! any-function!] "Function to apply, with eventual refinements."
    args    [block!] "Block of args, reduced first."

REFINEMENTS:
    /all    => Provides a continuous list of arguments, tail-completed with false/none.
    /safer  => Forces single refinement arguments, skip them when inactive instead of evaluating.

Notice that the func arg can now be a word! or path!, so you can use the name, or a path including refinements. That's right, the Dynamic Refinements feature explained above works with apply too. And having access to the name being used to call the function is enormously valuable when it comes to tracing and debugging. It's a huge win. 

One big difference is that all the arguments for apply are in a single block. Another is that you MUST include the on/off values for each dynamic refinement in the arg block, they DO NOT come from the environment (context). Compare this version to the one in the Dynamic Refinements section. Really, paste them into an editor and look at them side by side. 

>> only: false  dup: false  apply 'append/:only/:dup [[] [1] only dup 3] 
== [1]
>> only: true  dup: false  apply 'append/:only/:dup [[] [1] only dup 3] 
== [[1]]
>> only: false  dup: true  apply 'append/:only/:dup [[] [1] only dup 3] 
== [1 1 1]
>> only: true  dup: true  apply 'append/:only/:dup [[] [1] only dup 3] 
== [[1] [1] [1]]

; Refinement names in the arg block don't have to match the spec.
; You can use other names, or literal values. For example:

apply 'append/:only/:dup [[] [1] false false 3] 

a: b: false  apply 'append/:only/:dup [[] [1] a b 3]

It means you have to be more careful in lining things up with the function spec, in a different way than you're used to, but here's where you could use a computed refinement value, which may be useful for generative scenarios like testing. You can also see that both refinements are dynamic, so both need an associated on/off value in the args block.
>> only: does [random true]
== func [][random true]
>> blk: copy [] dup: no  loop 10 [apply 'append/:only/:dup [blk [1] only dup none]]
== [1 [1] 1 [1] [1] 1 [1] [1] 1 1]

But if you use a fixed refinement, it does not need the extra on/off value. In this example, /dup is always used, so there is no on/off value for it in the args, block, but its associated count arg has to be there, and is type-checked normally.

>> only: does [random true]
== func [][random true]
>> blk: copy [] dup: no  loop 10 [apply 'append/:only/dup [blk [1] only 2]]
== [[1] [1] 1 1 1 1 [1] [1] [1] [1] 1 1 [1] [1] [1] [1] 1 1 [1] [1]]
>> blk: copy [] dup: no  loop 10 [apply 'append/:only/dup [blk [1] only none]]
*** Script Error: append does not allow none! for its count argument
*** Where: append
*** Near : apply 'append/:only/dup [blk [1] only none]
*** Stack: 

But those aren't your only options. During the design of apply there was a lot of discussion about the interface(s) to it, and different use cases benefit from different models. For example, programmatically constructed calls means you need to build the path, and keep the args in sync. It may be easier to build a single block with everything in it, which you can do.

>> only: does [random true]
== func [][random true]
>> blk: copy []  loop 5 [apply :append [blk [1] /only only /dup no none]]
== [1 1 [1] [1] 1]
>> blk: copy []  loop 5 [apply :append [blk [1] /only only /dup yes 2]]
== [[1] [1] 1 1 1 1 [1] [1] 1 1]
>> blk: copy []  loop 5 [apply 'append [blk [1] /only only /dup no none]]
== [1 1 [1] [1] 1]
>> blk: copy []  loop 5 [apply 'append [blk [1] /only only /dup yes 2]]
== [[1] [1] [1] [1] 1 1 1 1 [1] [1]]

This interface is used if the first argument to apply is a function or lit-word, and /all is not used.

Apply/all

This is the most direct model, and what the others map to internally. In those models, you get friendly refinements, which may be optional, and those may have their own optional args. It's great for humans, and one of the best things about Redbol languages. But look at it from the view of a function spec.

>> print mold spec-of :append
[
    {Inserts value(s) at series tail; returns series head} 
    series [series! bitset! port!] 
    value [any-type!] 
    /part "Limit the number of values inserted" 
    length [number! series!] 
    /only {Insert block types as single values (overrides /part)} 
    /dup "Duplicate the inserted values" 
    count [integer!] 
    return: [series! port! bitset!]
]

The doc string isn't used when calling functions, so we can ignore that for this discussion. We can also ignore return: here. What's left is all the parameters (we often interchange arg and parameter, but there's a technical difference. Parameters are the named slots in a function spec, and arguments are the actual values passed in those slots). There are seven of those. Some are required args, some are refinements, and some are optional args. But there are seven slots, and when a function is invoked it expects there to be seven values on the stack that it could use if needed, or ignored if not.

When you use /all, you're telling apply that you are going to provide all those values in a single block, in the order of the function spec, like a stack frame (don't worry about the terminology too much if it's unfamiliar). Apply/all calls look like this:

1.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none false true 2]]
== [1 1 1 1 1 1 1 1 1 1]

2.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none true true 2]]
== [[1] [1] [1] [1] [1] [1] [1] [1] [1] [1]]

3.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none true false 2]]
== [[1] [1] [1] [1] [1]]

4.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none only false none]]
== [1 [1] 1 1 [1]]

5.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none 1 false none]]
*** Script Error: append does not allow integer! for its only argument
*** Where: append
*** Near : apply/all 'append [blk [1] false none 1 ]
*** Stack:  

6.
>> blk: copy []  loop 5 [apply/all 'append [blk [1] false none true true none]]
*** Script Error: append does not allow none! for its count argument
*** Where: append
*** Near : apply/all 'append [blk [1] false none true ]
*** Stack:  

7.
>> blk: copy []  loop 5 [apply/all 'append [blk [1]]]
== [1 1 1 1 1]

You can see that the refinement slots are now anonymous logic values in examples 1-3, but 4 uses only, our func from earlier examples, which randomly returns true or false. You can use anything that evaluates to logic! for a refinement slot. 5 shows that it has to be logic!, not just truthy, because types are checked (and logic refinement values then stand out against none argument values). And 6 shows that if you use /dup (second from the last arg), the count arg is also type checked, where 4 didn't complain because /dup was false. Confused yet? Look at 7. How can that work? I thought we had to fill all the slots! Yes and No. Apply "tail completes" the block with false/none values for you, if you don't provide enough arguments. Think of find, which has 16 slots. You only have to include enough args up to the last one you need. That may help, but if you need to use /match, the last refinement in find, you will have to provide all 16 args. Before you think this is unacceptable, consider our first repend example:

repend: func [
    {Appends a reduced value to a series and returns the series head} 
    series [series!] 
    value 
    /only "Appends a block value as a block"
][
    append/:only series reduce :value
]

It would look like this:

repend: func [
    {Appends a reduced value to a series and returns the series head} 
    series [series!] 
    value 
    /as-one "Appends a block value as a block"
][
    apply/all 'append [series reduce :value false none :as-one]
]

The point here is not to show that apply/all is longer, but that we can use a different name, where the first example must use :only in the path (make your own version to try it). Not all refinements will propagate using the same name. With /all it cares only about the logic value in the slot.

Apply/safer

/Safer is a form of short-circuit logic. Its purpose is to avoid the evaluation of unused args. Without it, everything in the args block is evaluated, but may be discarded if an associated refinement isn't active. The easiest way to explain this is with an example. 

; This is the function we're going to apply
applied: func [i [integer!] b /ref c1 /ref2 /ref3 c3 c4][
    reduce ['i i 'b b 'ref ref 'c1 c1 'ref2 ref2 'ref3 ref3 'c3 c3 'c4 c4]
]
; And some passive and active arg values
c: 0
bar40: does [4 * 2]
baz40: does [c: c + 1 456]

; Refinement args are evaluated
apply       'applied [10 "hi" /ref on bar40  /ref3 on  baz40            "ok"]

; No /safer difference because all refinement args are single values.
apply       'applied [10 "hi" /ref no bar40  /ref3 no  baz40            "ok"]
apply       'applied [10 "hi" /ref no bar40  /ref3 no  (c: c + 1 4 * 2) "ok"]
apply/safer 'applied [10 "hi" /ref no bar40  /ref3 no  (c: c + 1 4 * 2) "ok"]


apply       'applied [10 "hi" /ref no 4 * 2  /ref3 no  baz40            "ok"]
apply       'applied [10 "hi" /ref no bar40  /ref3 no  c: c + 1 4 * 2   "ok"]
apply/safer 'applied [10 "hi" /ref no 4 * 2  /ref3 no  baz40            "ok"]
apply/safer 'applied [10 "hi" /ref no bar40  /ref3 no  c: c + 1 4 * 2   "ok"]


In Real Life

Here are some examples of these new features being applied in the Red code base. The parse-trace example is jaw-dropping, not because it turns 9 lines into 1 (though, wow!), but because it makes the intent so much clearer and eliminates so much redundant code and the errors they can lead to. Not only that, it adds a capability! Before now you couldn't use both refinements together, i.e. parse-trace/case/part, but now you can.

Things we left out

The design of apply took many turns, with long and vigorous discussion and analysis. Many views and preferences were expressed, and which ultimately led to dynamic refinements as what we called straight sugar. That is, syntactic sugar at its sweetest. We knew /all had to be there, as that's what the others build on, but it was originally the default. We all eventually agreed that it shouldn't be, as it's the lowest level and likely the least directly used, though still vital for some use cases. Our problem was striking a balance between what would be most useful, with minimal overlap in use cases, and making the rules too complex to remember and get right. So a couple models didn't make the cut.

If Dynamic Refinements are straight sugar, the candy-wrapper version might be something like this, where you can still use a path, but only the args are in the block.

apply-args: function [
    "Make straight sugar call from semi-sweet model."
    fn [word! path!] "Get-word refinements come from context."
    args [block!]	 "Args only, no refinement values."
][
    ; Use temp result so we don't return any extra args
    do compose [res: (fn) (args)]
    res
]

>> apply-args 'append [[] [a]]
== [a]
>> apply-args 'append/only [[] [a]]
== [[a]]
>> only: false
== false
>> apply-arg 'append/:only [[] [a]]
== [a]
>> only: true
== true
>> apply-args 'append/:only [[] [a]]
== [[a]]

It's easy, but has quite a bit of overhead, because of compose and do. Remember, this amount of overhead only matters in loops running thousands of times at the very least, or in a real-time interactive interface. When in doubt, clock it. Write for humans to understand, and only optimize as needed (and after you know what's slow)

Another model is name-value args. That is, you provide a structure of arg names and their values, which is applied. It can make some code much clearer, but you also have to make sure the names match, so any refactoring of names will break code, which doesn't happen if args are positional. This is a bit involved, but it shows the power of Red. We'll use objects in our example, for a particular reason. That reason is values-of. The idea being that apply/all wants all the args, in order, and every slot filled. If your object matches the function spec, it's a perfect match. But making objects manually that way is error prone. So we'll use reflection for an object tailor-made for a given function.

Step 1: Find all the words in the spec. Remember it could have doc strings and arg types as well.

func-spec-words: function [
    "Get all the word-type values from a func spec."
    fn [any-function!]
    /as type [datatype!] "Cast results to this type."
][
    arg-types: make typeset! [word! lit-word! get-word! refinement!]
    parse spec-of :fn [
        ; If we want an apply-specific version of objects, we could
        ; denote refinements with a sigil for added clarity.
        collect [
            any [set w arg-types keep (either type [to type w][w]) | skip]
        ]
    ]
]

Step 2:  Make an object from that

func-spec-to-obj-proto: function [
    "Returns an object whose words match a function's spec."
    fn [any-function!]
    ; The idea here is that you can both preset values that are in the spec,
    ; and also extend the object with extra words, which will be ignored.
    /with args [block!] "APPLY/ALL ignores extra values."
][
    obj: construct/with any [args []]
    construct append func-spec-words/as :fn set-word! none
    ; Refinement values in APPLY/ALL calls MUST be logic!, not none!.
    foreach word func-spec-words :fn [
        if refinement? word [set in obj to word! word false]
    ]
    obj
]

Step Aside: Here's another approach, which combines steps 1 and 2, and lets you use a path for the function arg.

; Alt approach to func-spec-to-obj-proto, does NOT allow extending the spec.
make-apply-obj-proto: function [
    "Returns an object whose words match a function's spec."
    fn [any-function! word! path!]
    /with args [block!] "TBD: If APPLY doesn't ignore extra values, keys must be in spec."
][
    if path? :fn [refs: next fn   fn: first fn]        ; split path
    if word? :fn [
        name: fn                                       ; hold on to this for error report
        set/any 'fn get/any fn                         ; get func value
        if not any-function? :fn [
            do make error! rejoin ["FN argument (" name ") does not refer to a function."]
        ]
    ]                                                  ; get func value
    obj: construct append func-spec-words/as :fn set-word! none	; make object
    ; Refinement values in apply calls MUST be logic!, not none!.
    foreach word func-spec-words :fn [
        if refinement? word [set in obj to word! word false]
    ]
    if refs [foreach ref refs [obj/:ref: true]]        ; set refinements
    ; can't use obj/:key: if key is a set-word!
    if args [foreach [key val] args [set in obj key :val]]  ; set arg values
    obj
]

o-1: make-apply-obj-proto/with 'find/case/part/tail/skip/with [wild: "*?+" length: 10 size: 2]

Step 3. Using the object with APPLY

; We finally get here, and it's anticlimactic.
apply-object: func [
    "Call APPLY using an object's values as args."
    fn  [any-function!]
    obj [object!]
][
    apply/all :fn values-of obj
]

And an example call:

>> o-fctm: make-apply-obj-proto/with 'find/case/tail/match [series: [a b c] value: 'a]
== make object! [
    series: [a b c]
    value: 'a
    part: false
    length: none
    only: false
    case: true
    same: fal...
>> apply-object :find o-fctm
== [b c]

But you may see that this is verbose and inefficient, making a whole object just for a call like this. And you'd be right. It's just an example.

You don't want to recreate objects like this, especially in a loop. But you don't have to. You can reuse the object and just change the important values in it. This blog is getting long already, so we'll leave that as an exercise for the reader, or a question in community chat. And if you reuse the same object, the overhead is minimal.

We even talked about an idea whose time has not come, and is not guaranteed to work in the future. Here's the idea:

dyna-ref: func [p [path!]][
    res: make path! collect [
        keep p/1
        foreach val next p [
            case [
                get-word? val [if get :val [keep to word! val]]
                all [paren? val get-word? val/1] [if do next val [keep to word! val/1]]
                paren? val [do make error! "Sorry, has to be (get-word expr) for use with dyna-path."]
                'else [keep val]
            ]
        ]
    ]
    res
]

c: true
print mold dyna-ref 'a/b/:c/(:d true)
print mold dyna-ref 'a/b/:c/(:d false)
c: false
print mold dyna-ref 'a/b/:c/(:d true)
print mold dyna-ref 'a/b/:c/(:d false)

That's right, it's a dialected path! that builds a dynamic path. Crazy, right? You may know that while paths can currently contain parens, for Rebol compatibility, that feature may go away. It has deep design implications, but is also very handy at times. And while this isn't part of Red, it's another example of how Red lets us think off the beaten path.


Interpreter improvements

Dynamic refinements and function application are supported at interpreter level, for maximum efficiency and code reuse (mostly arguments fetching and type-checking). In addition to that, long standing issues and needed simplifications have been made in the interpreter code. Here is the changelog:

Function arguments cache entirely reimplemented:
  • Massively reduces the amount of code needed to manage the caches.
  • Simpler and faster cache design, O(1) lookup time for refinements in paths.
  • Adds a context! to native!, action! and routine!, to speed up word lookups.
  • Fixes long standing issue #4854 (CRASH & CORRUPTION in "dynamic" function calls with refinements)
Simpler reimplementation of op! datatype and its evaluation:
  • red-op! structure redesigned: it is now a shallow copy of the underlying function (nodes are not copied), the sub-type is stored in the op! header.
  • is infix function has been deprecated and replaced by a prefix version: relate.
  • Massive code reduction and simplification compared to previous version. Now op! maximally reuses the interpreter code for evaluating other functions.

Those changes lead to a general interpreter speed-up of 3-5% and up to 20% in some cases.


Additional language changes:
  • The in native now accepts any-function! as its first argument and refinements as the second argument. Refinements, if matched, will be converted to word values. This makes in a fast way to check if a symbol is part of an object or function spec and return a word bound to that context.

Conclusion

The most important thing you should do now is try it. It's a new design, and we want to hear what people like, see what they try, and where it falls short.


Happy Reducing!

-Red Team
Fork me on GitHub