tag:blogger.com,1999:blog-59361118377819350542024-03-17T14:07:44.516+01:00Red Programming LanguageNenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.comBlogger92125tag:blogger.com,1999:blog-5936111837781935054.post-32789584258943331572024-02-19T16:22:00.003+01:002024-02-19T16:37:37.611+01:000.6.5: Changelog<p>Bumping up the version number was motivated by the breaking syntax change <a href="https://www.red-lang.org/2024/02/important-change-switching-map-and.html" target="_blank">done recently</a>. We do not offer specific builds for a given version number anymore since we provide automatic builds (with builds history) on each new master commit. Though, the changelog for new version number changes will still be provided...and this one is pretty big as it covers about 5000 commits! Hope this will help users who might have missed some changes to catch up.</p><p>613 PRs were <a href="https://github.com/red/red/pulls?page=1&q=is%3Apr+is%3Aclosed+created%3A2018-11-22..2024-12-31" target="_blank">merged</a>, 2415 fix commits were pushed, among which 902 are <a href="https://github.com/red/red/issues?q=is%3Aissue+created%3A2018-11-22..2024-12-31+is%3Aclosed" target="_blank">closing issues</a> tracked on Github. The most notable new features and changes are listed below with eventual links to docs or previous blog posts describing or mentioning them:</p><h3 style="text-align: left;">Main new features</h3><ul style="text-align: left;"><li>New datatypes: <a href="https://github.com/red/docs/blob/master/en/datatypes/money.adoc" target="_blank"><span style="font-family: courier;">money!</span></a>, <span style="font-family: courier;">ref!</span>, <a href="https://www.red-lang.org/2023/08/subpixel-gui.html" target="_blank"><span style="font-family: courier;">point2D!</span></a>, <a href="https://www.red-lang.org/2023/08/subpixel-gui.html" target="_blank"><span style="font-family: courier;">point3D!</span></a>.</li><li>New codecs: <span style="font-family: courier;">Redbin</span>, <span style="font-family: courier;">JSON</span>, <span style="font-family: courier;">CSV</span></li><li>New high-performance run-time lexer with instrumentation support.(<a href="https://www.red-lang.org/2020/08/a-new-fast-and-flexible-lexer.html" target="_blank">blog</a>)(<a href="https://github.com/red/docs/blob/master/en/lexer.adoc" target="_blank">doc</a>)</li><li>Interpreter instrumentation support (debugger, tracer, profiler).(<a href="https://github.com/red/docs/blob/master/en/interp-events.adoc" target="_blank">doc</a>)</li><li>New powerful APPLY native, with deep interpreter support.(<a href="https://www.red-lang.org/2023/06/dynamic-refinements-and-function.html" target="_blank">blog</a>)</li><li>Dynamic refinements support.(<a href="https://www.red-lang.org/2023/06/dynamic-refinements-and-function.html" target="_blank">blog</a>)</li><li>Adds compress and uncompress natives with gzib, zlib and deflate algorithms support.</li><li>Adds <span style="font-family: courier;">gpio://</span> port with GPIO dialect for RaspberryPi.(<a href="https://www.red-lang.org/2019/07/gpio-port-for-raspberry-pi.html" target="_blank">blog</a>)</li><li>Adds TAB navigation support to View. (<a href="https://www.red-lang.org/2023/11/tab-navigation.html" target="_blank">blog</a>)</li><li>Adds raw strings syntax support.(<a href="https://github.com/red/docs/blob/master/en/datatypes/string.adoc#literal-syntax" target="_blank">doc</a>)</li><li>Swaps map! and construction syntax. (<a href="https://www.red-lang.org/2024/02/important-change-switching-map-and.html" target="_blank">blog</a>)</li><li>Hashtables are now used for fast lookups in contexts.</li><li>Custom <a href="https://www.exploringbinary.com/quick-and-dirty-floating-point-to-decimal-conversion/" target="_blank">dtoa</a> library <a href="https://github.com/red/red/blob/master/runtime/dtoa.reds" target="_blank">implementation</a> to load and form float values.</li><li>Standard library and garbage collector stability vastly improved.</li></ul><div><br /></div><div>Finished or almost finished features in branches::</div><ul style="text-align: left;"><li>Full IO ports with async support, including new <span style="font-family: courier;">IPv6!</span> datatype.(<a href="https://github.com/red/red/tree/IO" target="_blank">branch</a>)</li><li>TextUI backend to View.(<a href="https://github.com/red/red/pull/5378" target="_blank">PR</a>)</li><li>XML codec.(<a href="https://github.com/red/red/pull/5026" target="_blank">PR</a>)</li></ul><p></p><p><span style="font-size: medium;"></span></p><p><span style="font-size: medium;"></span></p><h3 style="text-align: left;">Other general new features or changes</h3><div><ul style="text-align: left;"><li>New natives: TRANSCODE, SCAN, AS-MONEY, ENHEX.</li><li>New functions: SINGLE?, LAST?, DT, TRANSCODE-TRACE, TRACE, CLOCK, NO-REACT, DO-NO-SYNC</li><li>New routines: SET-TRACE, TRACING?</li><li>Extends EMPTY? to support <span style="font-family: courier;">map!</span> values.</li><li>Allows NONE as value in <span style="font-family: courier;">map!</span>.</li><li>Adds REMOVE/KEY support for removing <span style="font-family: courier;">map!</span> entries.</li><li>Adds FOREACH support for <span style="font-family: courier;">map!</span>.</li><li>Add new-lines automatically when converting maps to blocks.</li><li>Supports <span style="font-family: courier;">issue!</span>,<span style="font-family: courier;"> money!</span> and <span style="font-family: courier;">refinement!</span> as key in map</li><li>Allows <span style="font-family: courier;">any-string!</span> series to be matched against binary input in PARSE.</li><li>Adds <span style="font-family: courier;">Accept-Charset</span> and <span style="font-family: courier;">User-agent</span> to HTTP header.</li><li>Update <span style="font-family: courier;">libcrypto </span>version requirement on Linux platforms.</li><li>Adds support for native port actors.</li><li>Extends ROUND/TO to support <span style="font-family: courier;">time!</span>.</li><li>Adds a cross-platform threading library to Red runtime library.</li><li>Adds a FIFO MPMC queue to Red runtime library.</li><li>Adds history support to Red console.</li><li>Extends GET native to accept <span style="font-family: courier;">any-word!</span> as argument.</li><li>FIND is now returning FALSE instead of NONE when used on bitsets.</li><li>Deprecates FIND on objects (redundant with IN).</li><li>Adds ABOUT function to console.</li><li>Preprocessor: <span style="font-family: courier;">fetch-next </span>now supports set-words/paths, get/lit-args, object/series paths.</li><li>COMPLEMENT is now allowed on tuples.</li><li>Adds RANDOM/SECURE.</li><li>SORT support of /skip & /all & /compare integer extended to strings, vectors and binaries.</li><li>Extends FIND/LAST to support <span style="font-family: courier;">hash!</span>.</li><li>Allows FINDing by datatype and typeset on <span style="font-family: courier;">hash!</span> series.</li><li>Removes percent-encoding from files, use double-quotes when needed instead.</li><li>Adds SYSTEM/CATALOG/ACCESSORS.</li><li>Allows <span style="font-family: courier;">pair!</span> for COPY/PART argument on series.</li><li>Allows LOOP and REPEAT to take a float argument as counter.</li><li>Extends REDUCE/INTO to support <span style="font-family: courier;">hash!</span> destination.</li><li>Extends BODY-OF to support <span style="font-family: courier;">action!</span> and <span style="font-family: courier;">native!</span> values.</li><li>REPLACE reimplemented.</li><li>Adds support for REVERSE/SKIP.</li><li>Extends VALUE? native supports <span style="font-family: courier;">any-word!</span> argument.</li><li>Enable <span style="font-family: courier;">image!</span> in red/core on Linux.</li><li>Adds GC support to libRed.</li><li>Extends DISTANCE? to support <span style="font-family: courier;">pair!</span> arguments.</li><li>Adds /KEEP refinement to TRY.</li><li>Implements RENAME action for FILE! datatype.</li><li>Adds routine arguments type-checking support to compiler.</li><li>Accelerates the output speed of LIST-DIR.</li><li>Allows <span style="font-family: courier;">error!</span> values to be modified by users.</li><li>Adds <span style="font-family: courier;">[trace]</span> and <span style="font-family: courier;">[no-trace]</span> function attributs.</li><li>Faster/simpler EMPTY? function implementation.</li><li>Forces the inclusion of Git information in the runtime.</li><li>Optimization for appending values into hashs.</li><li>Extends CHANGE to support <span style="font-family: courier;">image!</span> argument.</li><li>PICK on <span style="font-family: courier;">pair!</span>, <span style="font-family: courier;">date!</span> and <span style="font-family: courier;">time!</span> values now support named accessors as index argument.</li><li>Adds PICK action to <span style="font-family: courier;">event!</span> datatype.</li><li>Makes RECYCLE returns the allocated memory total by default.</li><li>Cleaner implementation of deep reactive paths support.</li><li>Internalizes SYSTEM/STATE/NEAR value.</li><li>Compound scalar datatypes (<span style="font-family: courier;">pair!</span>, <span style="font-family: courier;">date!</span>, <span style="font-family: courier;">time!</span> and <span style="font-family: courier;">tuple!</span>) will now emit ON-CHANGE events when one of their component is changed using an access path (both in compiled and interpreted code).</li><li>Adds reactivity support to <span style="font-family: courier;">bitset!</span> values.</li><li>Adds support for REFLECT action on <span style="font-family: courier;">bitset!</span> values.</li><li>Reports a proper path in compiled path errors.</li><li>Adds memory usage before/after a GC cycle in debug output.</li><li>Measures GC time in debug mode.</li><li>[ARM] Adds "division by zero" and "division overflow" checks in debug mode.</li></ul><h3 style="text-align: left;">Parse</h3><ul style="text-align: left;"><li>KEEP PICK on paren expressions now merges list of values to collected block.</li><li>Optimizes Parse's KEEP memory usage on strings/binaries.</li><li>Removes the end checking in iteration rules.</li><li>Speed optimizations for `TO <token>` rules.</li><li>Adds a fast path for `TO/THRU end` rules.</li><li>New set of optimizations for looping on a <span style="font-family: courier;">char!</span> value in PARSE.</li></ul></div><h3 style="text-align: left;">VID</h3><div><div><ul style="text-align: left;"><li>Smarter merging of actors in a style with custom actors in the instance.</li><li>Added <span style="font-family: courier;">password </span>flag for hidden input</li><li>Adds <span style="font-family: courier;">scrollable </span>flag.</li><li>defines VID <span style="font-family: courier;">origin </span>and <span style="font-family: courier;">spacing </span>values per backend.</li><li>adds <span style="font-family: courier;">next </span>and <span style="font-family: courier;">prev </span>as default options to VID-made faces.</li></ul></div></div><h3 style="text-align: left;">View</h3><div><ul style="text-align: left;"><li>Adds a calendar widget.</li><li>Adds support for tri-state checkboxes.</li><li>Significant GTK backend improvements and upgrades to match other backends.</li><li>Better handling of DPI changes.</li><li>Handles <span style="font-family: courier;">pause </span>and <span style="font-family: courier;">scroll-lock</span> keys.</li><li>Now EVENT/PICKED is a float wheel delta-value in <span style="font-family: courier;">on-wheel</span> event.</li><li>Improves user experience when closing window that contains a large number of faces.</li><li>Scale the font size with <span style="font-family: courier;">Ctrl</span> + mouse wheel in GUI console.</li><li>Adds <span style="font-family: courier;">stop-events</span> function to easily exit a View events loop.</li><li>Adds <span style="font-family: courier;">resize-ns</span> and <span style="font-family: courier;">resize-ew</span> mouse cursors.</li><li>Adds <span style="font-family: courier;">bounds </span>option to <span style="font-family: courier;">/options</span> facet for restricted dragging area.</li><li>Adds new <span style="font-family: courier;">/sync </span>refinement to VIEW function. </li><li>Adds <span style="font-family: courier;">/color</span> facet info to DUMP-FACE output.</li><li>Improves memory usage when changing <span style="font-family: courier;">/draw</span> facet content.</li><li>Adds support for semi-transparent no-border top-level windows (Windows).</li><li>Minimal dark mode support on GUI console.</li></ul></div><h3 style="text-align: left;">Draw</h3><div><ul><li>Switch command parameters to <span style="font-family: courier;">point2D!</span> and <span style="font-family: courier;">float!</span> for subpixel precision.(<a href="https://www.red-lang.org/2023/08/subpixel-gui.html" target="_blank">blog</a>)</li><li>New Direct2D backend for Windows.</li><li>Add <span style="font-family: courier;">line-pattern</span> command for drawing dashed lines.</li><li>Extends <span style="font-family: courier;">scale </span>command to support <span style="font-family: courier;">percent!</span> values.</li><li>Supports image mapping to arbitrary quadrilateral.</li><li>Removes <span style="font-family: courier;">matrix-order</span> command in DRAW.</li></ul></div><div><h3 style="text-align: left;">Red/System</h3><div><ul style="text-align: left;"><li>Adds subroutines to R/S functions.(<a href="https://www.red-lang.org/2020/08/redsystem-new-features.html" target="_blank">blog</a>)</li><li>Implements <span style="font-family: courier;">system/io/*</span> instrinsics for CPU I/O read/write instructions.</li><li>Adds<span style="font-family: courier;"> system/stack/push-all</span> and <span style="font-family: courier;">system/stack/pop-all</span> intrinsics.(<a href="https://www.red-lang.org/2020/08/redsystem-new-features.html" target="_blank">blog</a>)</li><li>Support for atomic operations using <span style="font-family: courier;">system/atomic/*</span> intrinsics.(<a href="https://www.red-lang.org/2020/08/redsystem-new-features.html" target="_blank">blog</a>)</li><li>Adds <span style="font-family: courier;">#inline </span>directive to R/S for including assembled binary code.</li><li>Implements support for integer hardware division (ARMv7+).</li><li>Generates optimized code for divisions by a power of 2 literal (ARM).</li><li>Vastly improved <span style="font-family: courier;">loop </span>counter handling robustness.</li><li>Full support for special float values (<span style="font-family: courier;">-0.0</span>, <span style="font-family: courier;">1.#NaN</span>, <span style="font-family: courier;">1.#INF</span>, <span style="font-family: courier;">-1.#INF</span>).</li><li>Drops support for <span style="font-family: courier;">%</span> and <span style="font-family: courier;">// </span>operators on float types.</li><li>Adds support for function pointers in literal arrays.</li><li>Switches to 16-bytes stack alignment on Linux.</li><li>Adds <span style="font-family: courier;">log-2</span> function to standard library.</li><li>Allows cross-referenced aliased fields in structs defined in same context.</li><li>Support multiple variable assignments.</li><li>Allows grouping arguments and local variables type specification.</li><li>Relax lexical format of hexadecimal literals.</li><li>Allows get-paths pointers on function nested in contexts.</li><li>Adds support for simple pointer to pointer type.</li><li>Allows <span style="font-family: courier;">function!</span> type to be specified for local variables.</li></ul></div></div><h3 style="text-align: left;">Toolchain</h3><div><ul style="text-align: left;"><li>Optimizes critical section in linker, twice faster linking time now on average.</li><li>Adds <span style="font-family: courier;">--show-func-map</span> compilation option.</li><li>Various minor improvements to PE format support.</li><li>Adds working <span style="font-family: courier;">Linux-musl</span> target.</li><li>Expands libRedRT to support View backend.</li><li>Adds<span style="font-family: courier;"> --no-view</span> option for Red binaries.</li><li>Shows the global words used by the toolchain in the compilation report.</li><li>Adds new Pico compilation target.</li></ul></div>Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com3tag:blogger.com,1999:blog-5936111837781935054.post-36192817644317960962024-02-11T15:37:00.004+01:002024-02-12T13:37:23.146+01:00Important Change! Switching map and construction syntax.<p>Sometimes deep changes take a huge amount of code. Sometimes they take a
lot of detailed explanation and consideration, leading to long discussions and
people taking sides. Rarely does an important syntactic change to a language
happen quickly, with universal agreement, simple implementation,
<i>and</i> tools to help update scripts in the wild. Today is one of
those rare days.
</p>
<p>
Admittedly, this <i>idea</i> has been discussed for a long time. It would
surface, people nodded their virtual heads, and it would submerge again. Today
it's ready to deploy. Not only that, but Rebol3 is making the same change, so
the two languages will still be compatible in this regard.
</p>
<p><span style="font-size: large;">What is the change?</span></p>
<p>
It's easy to describe. Today,
<span style="color: #6aa84f; font-family: courier;"><b>map!</b></span> values
use this syntax:
<span style="color: #6aa84f; font-family: courier;"><b>#(...)</b></span> and
construction syntax (sometimes called <i>serialized form</i> or
<i>loadable form</i>) looks like this:<span style="color: #6aa84f; font-family: courier;"><b> #[...]</b></span>. Going forward, those syntactic forms will be swapped. Why? The answer is
easy. In Redbol langs, blocks do not evaluate by default, you have to
<span style="color: #6aa84f; font-family: courier;"><b>do</b></span> or
<span style="color: #6aa84f; font-family: courier;"><b>reduce</b></span> them.
Parens, on the other hand, <i>do</i> evaluate by default. Today, maps use
paren-like syntax, but they do <i>not</i> evaluate, while construction
syntax uses block-like syntax, but <i>does</i> evaluate. This is a
carryover from Rebol, so the major concession here is that Red and Rebol3 will
no longer be compatible with Rebol2's construction syntax.
</p>
<p>
If you've never heard of construction syntax, there's a nice explanation of it
<a href="https://stackoverflow.com/questions/27070260/whats-the-purpose-of-datatype-constructor-in-rebol" target="_blank">here</a>. Red only supports a few values via construction syntax today, all datatype
literals, true, false, none, and unset; but eventually it will support much
more. If you look at the help for <span style="color: #6aa84f; font-family: courier;"><b>mold</b></span>, you'll see that <span style="color: #6aa84f; font-family: courier;"><b>/all</b></span> is TBD (very partially implemented for now), and
that's how you create loadable, serialized, data that can safely and easily
contain any value (like <span style="color: #6aa84f; font-family: courier;"><b>redbin</b></span> but readable by humans). It helps avoid cases
where
<span style="color: #6aa84f; font-family: courier;"><b>none</b></span> or
<span style="color: #6aa84f; font-family: courier;"><b>true/false</b></span> may load as words. This is also why
<span style="color: #6aa84f; font-family: courier;"><b>construct</b></span>
evaluates those specific words (including also <span style="color: #6aa84f; font-family: courier;"><b>on/off/yes/no</b></span>), but not others. When loading untrusted data,
we have to strike a balance between ease of use and safety.
</p>
<p><span style="font-size: large;">What do I have to do?</span></p>
<p>
Not much. There are two tools available, which will convert your scripts
automatically. The first is small and simple, showing just how powerful Red
is, and leveraging its lexer instrumentation. You can find it
<a href="https://github.com/red/red/blob/master/utils/migration/map-conv.red" target="_blank">here</a> (once merged, that branch may go away and the tool will be in the main
branch). The second is a more advanced and standalone tool written by
@hiiamboris. You can find that
<a href="https://codeberg.org/hiiamboris/red-cli/src/branch/master/mockups/mapconv" target="_blank">here</a>.
</p>
<p>
For the simple script it's <i><b>necessary</b></i> that you run it under a
current version of Red's lexer. Once the change is in place, running it under
the new lexer will make the exact opposite change. Of course, you can compile
it into a standalone EXE, or use Boris' app, which is already available.
</p>
<p>To run the simple script from a Red console, you can just:</p>
<pre class="code"> do https://raw.githubusercontent.com/red/red/master/utils/migration/map-conv.red
</pre>
<p>then you can use
<span style="color: #6aa84f; font-family: courier;"><b>help</b></span>
<b><span style="color: #6aa84f; font-family: courier;">map-conv</span></b> to see all the available options. By default it runs in preview mode, making no changes, and
showing you all the instances it found, which will be changed if you use the
<span style="color: #6aa84f; font-family: courier;"><b>/save</b></span>
refinement. A copy of each changed file is created with a <span style="font-family: courier;">.saved</span> extension in the same folder. If you don't want them, you can use the <span style="color: #6aa84f; font-family: courier;"><b>/no-copy</b></span> refinement.</p>
<p>
A word of warning, if you run the conversion tools a second time on the same
files, they will convert the data back, because they can't know what you're
thinking. On the bright side, this is an effective "undo" feature. Still, it's
wise to back up your data before running any tools against them.
</p>
<p>
Thanks to the power of Red, and these tools, there are already PRs pending for
updates to docs and community scripts. But the other thing you can do to help
is to let us know when you find things that need to be updated for this
change, and especially if you run into any issues when converting your own
code.
</p>
<p><span style="font-family: inherit; font-size: large;">Conclusion</span></p>
<p>
We know changes like this can be hard, but better now than when there is even
more code in the wild that would be affected. If we had been any other
language, this long-view improvement might not have happened. Only because Red
(and Redbol langs in general) can consume their own code as data and have
powerful parse and lexing features, was this change so easy and safe. It's
still a code-porting process, and if you run multiple versions of Red you may
need to maintain separate versions for a while. The other hard part is
retraining your hands and eyes to the new syntax.
</p>
<p>Happy Reducing!</p>
Unknownnoreply@blogger.com2tag:blogger.com,1999:blog-5936111837781935054.post-44587540662590591382023-11-22T23:00:00.001+01:002023-11-22T23:45:39.004+01:00Tab Navigation<p>
We finally got
<a href="https://en.wikipedia.org/wiki/Tabbing_navigation" target="_blank">tab navigation</a>
implemented! You might think it should have been an easy feature to add, but
achieving a consistent and controllable behavior across our different native
GUI backends is not that straightforward. So we opted for a mixed
implementation with a general high-level navigation layer in Red and left
<a href="https://en.wikipedia.org/wiki/Spatial_navigation" target="_blank">spatial navigation</a>
handling to each backend, in order to preserve the native behavior as much as
possible.
</p>
<p><span style="font-size: large;">Automatic navigation</span></p>
<p>
By default, pressing TAB key will allow you to navigate to all the GUI widgets
in a window, capable of acquiring the focus. Once the last widget is reached,
the next TAB press will circle back to the first focusable widget. Conversely,
back-navigation can be achieved using Shift-TAB key combination, circling from
first face to last one. Here is a simple example:
</p>
<pre class="code"> view [
text "Name" field focus return
text "Surname" field return
below
check "Single"
check "Employed"
button "Send"
]
</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSP672DqsfygeuPy2w0YtkQh1etv2inEENXX-8CceewJWfzyp6pKb1AwEEP89mbdyFA4iGWNgmJJ8exYk0lP34PBp5JFy7yN5TcHbLfYxIcB94-IWM5H3PmLHYnflBlSalfSDofZaZySmyQpVlyrw0H2Pt-LaawhxJpF9crHnb1otH37JWnL5w-SyXMgNO/s301/tab1.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="301" data-original-width="298" height="301" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgSP672DqsfygeuPy2w0YtkQh1etv2inEENXX-8CceewJWfzyp6pKb1AwEEP89mbdyFA4iGWNgmJJ8exYk0lP34PBp5JFy7yN5TcHbLfYxIcB94-IWM5H3PmLHYnflBlSalfSDofZaZySmyQpVlyrw0H2Pt-LaawhxJpF9crHnb1otH37JWnL5w-SyXMgNO/s1600/tab1.gif" width="298" /></a>
</div>
<p>
Note: check-boxes selection/unselection is done using the Space key (default
on Windows).
</p>
<p><br /></p>
<p>
It is possible to make a face "TAB-transparent", so that TAB navigation will
skip it in both directions. This is achieved by removing the
<span style="color: #6aa84f;"><b><span style="font-family: courier;">focusable</span> </b></span>flag from a navigable face. For example, in the following code, clicking on
the "Click me!" button will toggle the button's <b style="color: #6aa84f;"><span style="font-family: courier;">focusable</span> </b>flag on and off (using
<span style="color: #6aa84f; font-family: courier;"><b>set-flag/toggle</b></span>):
</p>
<pre class="code"> view [
text "Name" field focus return
text "Surname" field return
below
check "Single"
check "Employed"
button "Send"
button "Click me!" 100 [
face/text: pick ["TAB ignore" "TAB stop"] to-logic face/flags
set-flag/toggle face 'focusable
]
]
</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiz6KH_wGYaOHkipTGplsV2hKYrIESZ3KJSYm98FE4EJJ9tI6gLtG4tDF90bhpOhSKlbXfuggZDYIvVkcE7FNXSomlXqiGBpCuEcT2_I6GiXlXlq4RX6ktsMOo_i7yp4xu25EK7wboKlAeELoIQlBEVeZVc7q01If1gZrXkGbBRrz77mLg3J_3znUUjQ2vF/s354/tab2.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="354" data-original-width="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiz6KH_wGYaOHkipTGplsV2hKYrIESZ3KJSYm98FE4EJJ9tI6gLtG4tDF90bhpOhSKlbXfuggZDYIvVkcE7FNXSomlXqiGBpCuEcT2_I6GiXlXlq4RX6ktsMOo_i7yp4xu25EK7wboKlAeELoIQlBEVeZVc7q01If1gZrXkGbBRrz77mLg3J_3znUUjQ2vF/s16000/tab2.gif" /></a>
</div>
<br />
<p><br /></p>
<p>
In case of
<b><span style="color: #6aa84f; font-family: courier;">area </span></b>face, the default behavior for TAB navigation means that tab characters
cannot be input in the area. In such cases, the alternative Ctrl-TAB key
combination can be used to input tab characters. In case the <b style="color: #6aa84f;"><span style="font-family: courier;">focusable</span> </b>flag is removed from an area face, then TAB key will directly produce tab
characters. Here is an example:
</p>
<pre class="code"> view [
text "Name" field focus return
text "Surname" field return
below
text "Comments" com: area
button "Send"
button "Toggle Area" [set-flag/toggle com 'focusable]
]
</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGmReuDpyW3ziRvmV6qZkQjCewnLGFhAPz0EAuv2sqyqRXlOA_hR-DK9WD2OjuzBPxsE3DFam7XyCVjbpqJ8Wst7hqYuPT-TFsVqQuAFgYyWm4Xwjpk2SSldsUp6IZKtmCDjx_hbOIXlJvL0DCSHP5Kes-3JBQ9WU0aRuP-9P9uaazLaFdRNSY841rgexE/s547/tab3.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="547" data-original-width="299" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjGmReuDpyW3ziRvmV6qZkQjCewnLGFhAPz0EAuv2sqyqRXlOA_hR-DK9WD2OjuzBPxsE3DFam7XyCVjbpqJ8Wst7hqYuPT-TFsVqQuAFgYyWm4Xwjpk2SSldsUp6IZKtmCDjx_hbOIXlJvL0DCSHP5Kes-3JBQ9WU0aRuP-9P9uaazLaFdRNSY841rgexE/s16000/tab3.gif" /></a>
</div>
<p>
Note: when the <b style="color: #6aa84f;"><span style="font-family: courier;">focusable</span></b> flag is on, Ctrl-TAB is used to input tab characters, when it's off,
it's just using TAB key.
</p>
<p><br /></p>
<p><span style="font-size: x-large;">Manual override</span></p>
<p>
In some cases, the user can decide to set a different path for keyboard
navigation. For each navigable face (the ones with a <b style="color: #6aa84f;"><span style="font-family: courier;">focusable</span></b> flag), it is possible to manually define the next and/or previous one
when tabbing forth and/or back. In order to do so,
<span style="color: #6aa84f; font-family: courier;"><b>next </b></span>and
<span style="color: #6aa84f; font-family: courier;"><b>prev </b></span>options
can be set to define how tabbing will navigate to the next or previous face.
</p>
<p>
Here is a simple example where the default navigation is changed to jump into
fields marked as invalid or empty (using pink background) after a typical form
submission:
</p>
<pre class="code"> view [
group-box 2 [
style error: field pink
text "Name" field "John"
text "Surname" field "Smith"
text "Age" error "abc" focus
text "Address" error "-"
text "Zip code" field "12345"
text "City" error
text "Country" error
] return
btn-send: button "Send"
do [
list: collect [foreach-face self [if face/color = pink [keep face]]]
forall list [list/1/options/next: list/2]
btn-send/options/next: list/1
]
]
</pre>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgU4oJ9I0tgZj6-Iye7qYXeflH3pKNKIhelwoGNlq2m5J_gZdVnb4T8rDcCmCzjf-O3Saxt-RMvOLPa9BzDBLF7k_bQE9ICWvw5UyZtaTyF9wncin0HhqppEuK2hLFoHeU7Kcs7vu84SYi9jb9mRw3DnVm45-20J7KIOnXdLObDeLuaWt67QxN6TAD4Z84z/s503/tab4.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="503" data-original-width="346" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgU4oJ9I0tgZj6-Iye7qYXeflH3pKNKIhelwoGNlq2m5J_gZdVnb4T8rDcCmCzjf-O3Saxt-RMvOLPa9BzDBLF7k_bQE9ICWvw5UyZtaTyF9wncin0HhqppEuK2hLFoHeU7Kcs7vu84SYi9jb9mRw3DnVm45-20J7KIOnXdLObDeLuaWt67QxN6TAD4Z84z/s16000/tab4.gif" /></a>
</div>
<p>Notes:</p>
<p></p>
<ul style="text-align: left;">
<li>
For the sake of simplicity in this example, only forward navigation is
restricted, backward navigation will visit all focusable faces.
</li>
<li>
In the <span style="color: #6aa84f; font-family: courier;"><b>do</b></span> block,
<span style="color: #6aa84f; font-family: courier;"><b>self</b></span> refers to the window face, as <span style="color: #6aa84f; font-family: courier;"><b>do</b></span> denotes a global section, not widget-related.
</li>
<li>
When <span style="color: #6aa84f; font-family: courier;"><b>list/1</b></span> refers to the last element,
<span style="color: #6aa84f; font-family: courier;"><b>list/2</b></span> returns
<span style="color: #6aa84f; font-family: courier;"><b>none</b></span>, so it does not point to any specific face. In such case, the default tab
navigation will automatically (and conveniently) select the next face, which
is the "Send" button.
</li>
<li>
The last line is there to connect that last face (the button) to the first
face in our restricted list.
</li>
</ul>
<p></p>
<p><br /></p>
<p><span style="font-size: large;">Other notable changes</span></p>
<p>An important change concerns the
<span style="color: #6aa84f; font-family: courier;"><b>insert-event-func</b></span> function specification, it now requires a name as argument:
</p>
<pre class="code"> >> ? insert-event-func
USAGE:
INSERT-EVENT-FUNC name fun
DESCRIPTION:
Adds a function to monitor global events. Returns the function.
INSERT-EVENT-FUNC is a function! value.
ARGUMENTS:
name [word!]
fun [block! function!] "A function or a function body block."
</pre>
<p>
<br />
The name is an arbitrary word that only needs to be unique, so it becomes
easier to check if a given global handler has been installed or not. It also
makes it easier to remove it, as it can be referred by name in
<span style="color: #6aa84f; font-family: courier;"><b>remove-event-func</b></span>. Existing handler names can be checked using:
</p>
<pre class="code"> >> extract system/view/handlers 2
[tab field-sync reactors radio enter debug dragging]
</pre>
<p>Please update your code if you have been using those functions.</p>
<p>Enjoy!</p>
Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com4tag:blogger.com,1999:blog-5936111837781935054.post-43053363815624271062023-08-09T15:32:00.003+02:002023-08-09T17:49:24.204+02:00Subpixel GUI<p>
Maybe you didn't notice, but Red/View, our GUI engine, has subpixel precision
from the beginning! Unfortunately, that level of precision was not directly
accessible to end users, until now.
</p>
<p>
Actually, it would be more accurate to say that we had <a href="https://en.wikipedia.org/wiki/Sub-pixel_resolution" target="_blank">subpixel resolution</a> only so far. The guilty part is the
<span style="font-family: courier;">pair!</span><span style="font-family: inherit;"> datatype being limited to integer components only, while subpixel
precison requires decimal numbers. So we have recently introduced new
datatypes to cope with that.</span>
</p>
<p>
What urged us to make those changes now was a very peculiar visual glitch
caused by that dissonance. That glitch happens during face dragging
operations. Here is an example using our View
<a href="https://github.com/red/red/blob/master/tests/view-test.red" target="_blank">test script</a>:
</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBlZCIqAJqy8lOhvwXT_49m974ocbb9xJwfgHw4oGr4_yGr_w4aBfUQF4HIZHoZCpqgHQPAhoPIugfinmNuZZ_wrEN21ps9pRAXmt8aH0ZwxH8N8hvv0Yp8JxEalAP3Wrf1jBxMZUr8Tj1WeloAgwKCbSbDWp3oUSabKybi0RYVa3FE-Yj0VE5qx5pW_XQ/s551/glitch.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="441" data-original-width="551" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBlZCIqAJqy8lOhvwXT_49m974ocbb9xJwfgHw4oGr4_yGr_w4aBfUQF4HIZHoZCpqgHQPAhoPIugfinmNuZZ_wrEN21ps9pRAXmt8aH0ZwxH8N8hvv0Yp8JxEalAP3Wrf1jBxMZUr8Tj1WeloAgwKCbSbDWp3oUSabKybi0RYVa3FE-Yj0VE5qx5pW_XQ/s16000/glitch.gif" /></a>
</div>
<p>
As you can see, on some positions, the face starts <i>shaking</i> while the
mouse cursor remains still. This affects any type of face. The shaking is
about ±2 pixels. It is caused by the difference in precision between the
<span style="color: #ffa400; font-family: courier;">/offset</span> facet
expressed in integer numbers and the backend API, which only deals with
floating point numbers. The accumulated error when converting
integer->float->integer gives a 2 pixels difference. Such error happens
on displays where the scaling factor is different from 100%. With the rise of
2K, 3K and 4K displays, a scaling factor > 100% has become the norm, making
this glitch more frequent. You might think that this is not a big issue until
you start building custom scrollbars and see your entire scrolled content
shaking massively...
</p>
<h3 style="text-align: left;">New point datatypes</h3>
<p>
In order to provide decimal positions and sizes for View faces, extending the
existing <span style="font-family: courier;">pair!</span> datatype was
considered, though, the pair syntax can hardly scale up for such needs:
</p>
<pre class="code"> 2343.122x54239.44
2343.122x54239.44x6309.332
2343.122x54239.44x6309.332x442.3321
2.33487e9x54239.44
2.33487e9x54239.44x9.83242e17
2.33487e9x54239.44x9.83242e17x5223.112
1.#infx1.#infx1.#inf
</pre>
<p>
As you can notice there, it quickly becomes difficult to read and identify the
individual components. So we opted for adding a new literal form (hence a new
datatype) that matches how coordinates for two or more dimensions are commonly
represented:
</p>
<pre class="code"> (2343.122, 54239.44)
(2343.122, 54239.44, 6309.332)
(2343.122, 54239.44, 6309.332, 442.3321)
(2.33487e9, 54239.44)
(2.33487e9, 54239.44, 9.83242e17)
(2.33487e9, 54239.44, 9.83242e17, 5223.112)
(1.#inf, 1.#inf, 1.#inf)
</pre>
<p>
Such literal forms requires the comma character to be a delimiter, so that it
cannot be used anymore as a decimal separator. That was, unfortunately, a
necessary decision in order to unlock such literal forms. The gains should be
bigger than the loss.
</p>
<p>So, two new datatypes have been added:</p>
<p></p>
<ul style="text-align: left;">
<li>
<span style="color: #ffa400; font-family: courier;">point2D!</span>: a
two-dimensional coordinate or size.
</li>
<li>
<span style="color: #ffa400; font-family: courier;">point3D!</span>: a
three-dimensional coordinate or size.
</li>
</ul>
<p></p>
<p>Their canonical lexical forms are:</p>
<pre class="code"> (<x>, <y>)
(<x>, <y>, <z>)
where <x>, <y> and <z> are integer or float numbers.
</pre>
<p>
Optional spaces are allowed anywhere inside the point literals on input, they
will be removed on loading.
</p>
<pre class="code"> >> (1,2)
== (1, 2)
>> ( 1.35 , 2.4 )
== (1.35, 2.4)
</pre>
<p>
Both for 2D and 3D points, their components are internally stored as 32-bit
floating point numbers, so that their precision is limited to 7 digits. This
should be far enough for their use-cases though.
</p>
<p>
When one of the components has a fractional part equal to zero, it is displayed
without the <span style="font-family: courier;">.0</span> part for easier
reading. Similarly, integers are accepted as input for any component and are
internally converted to a 32-bit float.
</p>
<pre class="code" style="text-align: left;"> >> (0.3, 0.5) + (0.7, 0.5)
== (1, 1)
>> (2.0, 3.0)
== (2, 3)
</pre>
<h4 style="text-align: left;">Creation</h4>
<p>
Besides literal points, it is possible to create them dynamically, the same
way as pairs, using
<span style="color: #ffa400; font-family: courier;">make</span>,
<span style="color: #ffa400;"><span style="font-family: courier;">to</span> </span>or one of the
<span style="color: #ffa400; font-family: courier;">as-*</span> native
functions:
</p>
<pre class="code"> >> make point2D! [2 4.5]
== (2, 4.5)
>> to-point2D 1x2
== (1, 2)
>> as-point3D 1 (3 / 2) 7 * 0.5
== (1, 1.5, 3.5)
</pre>
<h4 style="text-align: left;">Accessors</h4>
<p>
Point components can be individually accessed using ordinal numbers or
component names using action accessors or path syntax:
</p>
<pre class="code"> >> pick (2, 4.5) 1
== 2.0
>> pick (2, 4.5) 'y
== 4.5
>> p: (2, 4.5)
== (2, 4.5)
>> p/x
== 2.0
>> p/y: 3.14159
== 3.14159
>> p
== (2, 3.14159)
</pre>
<h4 style="text-align: left;">Math operations</h4>
<p>Basic math operations are supported as well:</p>
<pre class="code"> >> (1, 1) + (2, 3.5)
== (3, 4.5)
>> (1, 1) - (2, 3.5)
== (-1, -2.5)
>> (2, 3) * (10, 3.5)
== (20, 10.5)
>> (20, 30) / (10, 3)
== (2, 10)
</pre>
<p>
Notice that mixing pairs with point2D in math expressions is allowed. The pair
value will be promoted to a point2D in such case (as integers with floats):
</p>
<pre class="code"> >> 1x1 + (2, 3.5)
== (3, 4.5)
</pre>
<h4 style="text-align: left;">Other actions/natives</h4>
<pre class="code"> >> round (2.78, 3.34)
== (3, 3)
>> round/down (2.78, 3.34)
== (2, 3)
>> random (100, 100)
== (53, 81)
>> zero? (0, 0)
== true
>> min (10, 100) (24, 35)
== (10, 35)
>> max 10x100 (24, 35)
== (24, 100)
</pre>
Notice that pairs will be promoted to point2D in mixed use cases with
<span style="color: #ffa400; font-family: courier;">min</span>/<span style="color: #ffa400; font-family: courier;">max</span>.
<div>
<br />
<h3 style="text-align: left;">View and VID adjustments</h3>
<p>The main changes are in face! object:</p>
<p></p>
<ul style="text-align: left;">
<li>
<span style="color: #ffa400; font-family: courier;">/offset</span>: now
only accepts point2D! values.
</li>
<li>
<span style="color: #ffa400; font-family: courier;">/size</span>: accepts
both pair! and point2D! values.
</li>
</ul>
<p></p>
<p>
In VID, both pair and point2D values can be used to denote positions and
sizes, so that VID is backward compatible. All previous VID code should work
without any change. VID will convert all positions to point2D values. Sizes
by default in VID, keep using pairs, unless a point2D is provided by the user.
</p>
<p>
All Draw commands that were accepting pairs now also accept point2D values
for higher precision.
</p>
<p>
The related documentation will get updated soon to reflect those changes.
</p>
<p>
In order to illustrate the difference in using pairs and point2D positions,
here is a (not so) simple animation comparison showing the subpixel positioning
difference (correctness of animation in this case is privileged over simplicity of code):
</p>
<pre class="code"> view/no-wait [
size 800x200 space 0x0
b1: box 2x40 red return
b2: box 2x40 blue
]
x: b1/offset/x
until [
do-events/no-wait ; processes GUI events in queue
wait 0.1 ; slows down the animation
do-no-sync [ ; switches to manual faces redrawing
b1/offset/x: b1/offset/x + 0.1
b2/offset/x: to-integer x: x + 0.1
if all [b1/state b2/state][show [b1 b2]] ; redraws both faces
]
any [b1/offset/x > 700 none? b1/state none? b2/state]
]
</pre>
<p>Here's the zoomed capture of the result (on a display with 200% scaling factor):</p>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgVPWM8XAMRMSeFKPbcPpB5Un2AHE2I6iAr15WnD-UDDAz08xq459j-IncjXAFiNC6yDqu99u-LJ2w7Rhfh1LaWwkr54Fjl4sYLT2S_BmBSTBM-pLbyoMP9JCtA4oqNf2Sb6n0JnNNoKf9QmcqbHTru34Knt0FZ9E2BEg8tCuvSIC5ycV7rWXcJSCLFjI-/s796/run2.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="518" data-original-width="796" height="416" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhgVPWM8XAMRMSeFKPbcPpB5Un2AHE2I6iAr15WnD-UDDAz08xq459j-IncjXAFiNC6yDqu99u-LJ2w7Rhfh1LaWwkr54Fjl4sYLT2S_BmBSTBM-pLbyoMP9JCtA4oqNf2Sb6n0JnNNoKf9QmcqbHTru34Knt0FZ9E2BEg8tCuvSIC5ycV7rWXcJSCLFjI-/w640-h416/run2.gif" width="640" /></a>
</div>
<div><br /></div>The red bar uses the newly enabled subpixel precision, while the blue bar simulates the old pair positioning precision (so only allowing integer positions). What you can see is that the red bar makes two smaller steps while the blue bar makes a single one, looking more "jumpy".<br />
<p>This means that now animations on displays with a scaling factor > 100% can be smoother as they benefit from more accurate positioning.</p><p><u>Note</u>: the animation code above is far from being simple or elegant, we'll be working on improving that. The animation code could have been quite simpler by using a <span style="font-family: courier;">rate </span>option in VID and putting the animation code in a <span style="font-family: courier;">on-time</span> handler. Though, timer events firing (especially on Windows) are not very reliable, so unrolling a custom event loop lowers that risk when the timing is critical (like for fast game loops).</p><p><br /></p><p>As a conclusion, here is an old-school style starfield <a href="https://github.com/red/code/blob/master/Scripts/starfield.red" target="_blank">code demo</a> using 2D and 3D points (moving mouse left/right changes the stars speed):</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgv67jMw_YEHeCq-WZmd1lw6b5xs2SuO4a4kBFxdF1ZD4GuqRFKXBZSJBOb_XJfg8-qNSTfFthtKmz7hIza0hx5LgPgBMc-Sjn3UqthW9DCja5rtzIHxZdslT-KaAs5aFV19WOJHGG9fJUoRtgfUk-HogZVHAcV5Ntj2kbenZunc3w9DxfgigagBXTKR0EK/s1402/starfield.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1402" data-original-width="1372" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgv67jMw_YEHeCq-WZmd1lw6b5xs2SuO4a4kBFxdF1ZD4GuqRFKXBZSJBOb_XJfg8-qNSTfFthtKmz7hIza0hx5LgPgBMc-Sjn3UqthW9DCja5rtzIHxZdslT-KaAs5aFV19WOJHGG9fJUoRtgfUk-HogZVHAcV5Ntj2kbenZunc3w9DxfgigagBXTKR0EK/w626-h640/starfield.gif" width="626" /></a></div>
<p>Let us hear your feedback about those changes on our Gitter (now Matrix) <a href="https://app.gitter.im/#/room/#red_red:gitter.im" target="_blank">channel</a>.</p><p>Enjoy!</p>
</div>
Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com5tag:blogger.com,1999:blog-5936111837781935054.post-27230649354299245612023-06-08T00:55:00.006+02:002023-06-10T18:16:22.029+02:00Dynamic Refinements and Function Application<h3 style="text-align: left;">It's Time to Apply Yourself to Red</h3>
<br />
<div>
<span>Red has never had an
<span style="color: #f1c232;">apply</span> function, though we
knew it would come someday. In the meantime, some of us rolled our own.
Gregg's was simple, neither flexible nor efficient, and just a couple lines
of code. Boris made a much more capable version, but it could still only be
so fast as a mezzanine. R2 had a mezz version, which suffered the same
problem. All that changes now. <span style="color: #f1c232;">Apply</span> is dead! Long live <span style="color: #f1c232;">Apply</span>! It
required deep work, and a lot of design effort, but we think you'll like the
results, whether you're a high-level Reducer, or anxious to see how much
leverage you can, um, apply, from a functional perspective. Everybody
wins.
</span>
</div>
<br />
<div>
If you don't know what <span style="color: #f1c232;">apply</span> is,
in terms of functional languages, take a moment and <a href="https://en.wikipedia.org/wiki/Apply" target="_blank">read up</a>.
If you can get through the introduction there without getting dizzy, great.
If your head is spinning, feel free to stop after the first section of
this article and ignore the deep dive. You still get 90% of the value for
most high level use cases. Gregg got so dizzy that he fell down, but was
still able to help with this article.
</div>
<br />
<div>
<span>Function application is largely about composition. How you can combine
functions in a concise way for maximum leverage and minimum code. The
problem with its design in many languages is that it makes things harder to
understand. Rather than concrete functions names, there is indirection and
abstraction. It can be tricky to get right, especially in a flexible
language like Red, while also maintaining as much safety as possible. You
can drive fast, but still wear your seat belt.</span>
</div>
<br />
<h2 style="text-align: left;">Dynamic Refinements</h2>
<br />
<div style="text-align: left;">
<span style="font-family: inherit;">This subtle feature is likely to see wide use, because it will reduce code
and let people build more flexible functions. It's also easy to explain.
Here's an example. First, how you would write it today:</span>
</div>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">repend: func [
{Appends a reduced value to a series and returns the series head}
series [series!]
value
/only "Appends a block value as a block"
][
either only [
append/only series reduce :value
][
append series reduce :value
]
]</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">With dynamic refinements, it can be done like this.</span></div><br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">repend: func [
{Appends a reduced value to a series and returns the series head}
series [series!]
value
/only "Appends a block value as a block"
][
append/:only series reduce :value
]</span></code></pre>
<div style="text-align: left;"><span style="font-family: inherit;"><br />In case you missed the subtlety, it's </span><span style="color: #f1c232;">:only</span><span style="font-family: inherit;"> being a get-word in the path. That's right, it's evaluated, rather than being treated literally, just like you use in selector paths. The value for the dynamic refinement is taken from its context, which can be a refinement in the function, or a local value. It can be any truthy value to use the refinement, and it is only retrieved not evaluated. That means you can't use a computed value directly, which makes it safer. Other than that, paths work just as they always have. If a refinement is used, fixed (literal) or dynamic, which can be mixed however you want, any arguments it requires will be fetched and used. Otherwise they are silently ignored, so you don't have to clutter your code or worry about what a dynamic path expression will consume.<br /><br /></span></div>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">repend: func [
{Appends a reduced value to a series and returns the series head}
series [series!]
value
/only "Appends a block value as a block"
/dup "Duplicate the value"
count [integer!]
][
append/:only/:dup series reduce :value count
]
>> only: false dup: false repend/:only/:dup [] [1] 3
== [1]
>> only: true dup: false repend/:only/:dup [] [1] 3
== [[1]]
>> only: false dup: true repend/:only/:dup [] [1] 3
== [1 1 1]
>> only: true dup: true repend/:only/:dup [] [1] 3
== [[1] [1] [1]]
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">This is an incredibly exciting and powerful feature. It's a shame there isn't more to explain. ;^)</span></div>
<br />
<h2 style="text-align: left;">Function Application</h2>
<br />
<div style="text-align: left;">
<span>Functional Programming has never yet become mainstream, though it has
periodic rises in popularity and a devoted following in many language camps.
Even Red. Yes, Red is a functional language. It's not a
<i>pure</i> functional language, because functions can have side effects,
but functions are first class values and can be passed around like any
other. It lets you do things like this:</span>
</div>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> do-math-op: func [
fn [any-function!]
arg-1 [number!]
arg-2 [number!]
][
fn arg-1 arg-2
]
== func [fn [any-function!] arg-1 [number!] arg-2 [number!]][fn arg-1 arg-2]
>> do-math-op :add 1 2
== 3
>> do-math-op :subtract 1 2
== -1
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">That's called a Higher Order Function, or HOF. It also means you can <i>return</i> a function as a result.</span></div>
<br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> make-multiplier: func [
arg-1 [number!]
][
;; We're making a function and returning it here.
func [n [number!]] compose [(arg-1) * n]
]
== func [arg-1 [number!]][func [n [number!]] compose [(arg-1) * n]]
>> fn-m: make-multiplier 4
== func [n [number!]][4 * n]
>> fn-m 3
== 12</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
That's all well and good, but what if you want to call different functions
that take a different number or type of arguments? Now it gets tricky, and
inefficient. Because Red uses free-ranging evaluation (function args are not
contained in a paren or marked as ending in any way at the call site), how
do you handle different arities (number of arguments)? Here's a very simple
</span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> mezzanine:</span>
</div>
<br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">apply: func [
"Apply a function to a block of arguments."
fn [any-function!] "Function to apply"
args [block!] "Arguments for function"
/only "Use arg values as-is, do not reduce the block"
][
args: either only [copy args] [reduce args]
do head insert args :fn
]</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
So easy! The power of Red. But there is a cost. It's a mezzanine function, so
it's slower than a native function, the args are copied or reduced, and then
</span><span style="color: #f1c232;">do</span><span style="font-family: inherit;"> is used to evaluate it. We can live with this kind of overhead for a
great deal of Red code, but </span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> is a building block, and may be used in
deep code where performance is important. You may also have noticed that the
</span><span style="color: #f1c232;">fn</span><span style="font-family: inherit;"> argument is </span><span style="color: #f1c232;">any-function!</span><span style="font-family: inherit;">, that means two things: 1) If you want to
know the <i>name</i> of the function, the word that refers to it, too bad.
You'd have to pass another arg for that. 2) Refinements. You can't use them
with this model. And that limitation is a killer. For example, you could pass
</span><span style="color: #f1c232;">:append</span><span style="font-family: inherit;"> but not </span><span style="color: #f1c232;">:append/:only</span><span style="font-family: inherit;">. And there's no way you could have an
</span><span style="color: #f1c232;">/only</span><span style="font-family: inherit;"> refinement in <i>your</i> function and just pass that along. Until
now. </span>
</div>
<br />
<h3 style="text-align: left;">The Real Apply</h3>
<br />
<div style="text-align: left;">
<p>Here's the new <span style="color: #f1c232;">apply</span> native that is now available in Red:</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">USAGE:
APPLY func args
DESCRIPTION:
Apply a function to a reduced block of arguments.
APPLY is a native! value.
ARGUMENTS:
func [word! path! any-function!] "Function to apply, with eventual refinements."
args [block!] "Block of args, reduced first."
REFINEMENTS:
/all => Provides a continuous list of arguments, tail-completed with false/none.
/safer => Forces single refinement arguments, skip them when inactive instead of evaluating.
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
Notice that the </span><span style="color: #f1c232;">func</span><span style="font-family: inherit;"> arg can now be a </span><span style="color: #f1c232;">word!</span><span style="font-family: inherit;"> or </span><span style="color: #f1c232;">path!</span><span style="font-family: inherit;">, so you can use the
name, or a path including refinements. That's right, the Dynamic Refinements
feature explained above works with </span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> too. And having access to the name
being used to call the function is enormously valuable when it comes to
tracing and debugging. It's a huge win.
</span></div>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
One big difference is that all the arguments for </span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> are in a single
block. Another is that you <b>MUST</b> include the on/off values for each
dynamic refinement in the arg block, they <b>DO NOT</b> come from the
environment (context). Compare this version to the one in the Dynamic
Refinements section. Really, paste them into an editor and look at them side
by side.
</span></div>
<br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> only: false dup: false apply 'append/:only/:dup [[] [1] only dup 3]
== [1]
>> only: true dup: false apply 'append/:only/:dup [[] [1] only dup 3]
== [[1]]
>> only: false dup: true apply 'append/:only/:dup [[] [1] only dup 3]
== [1 1 1]
>> only: true dup: true apply 'append/:only/:dup [[] [1] only dup 3]
== [[1] [1] [1]]
; Refinement names in the arg block don't have to match the spec.
; You can use other names, or literal values. For example:
apply 'append/:only/:dup [[] [1] false false 3]
a: b: false apply 'append/:only/:dup [[] [1] a b 3]
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
It means you have to be more careful in lining things up with the function
spec, in a different way than you're used to, but here's where you could use a
computed refinement value, which may be useful for generative scenarios like
testing. You can also see that both refinements are dynamic, so both need an
associated on/off value in the args block.
</span></div>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> only: does [random true]
== func [][random true]
>> blk: copy [] dup: no loop 10 [apply 'append/:only/:dup [blk [1] only dup none]]
== [1 [1] 1 [1] [1] 1 [1] [1] 1 1]
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
But if you use a fixed refinement, it does not need the extra on/off value. In
this example, </span><span style="color: #f1c232;">/dup</span><span style="font-family: inherit;"> is always used, so there is no on/off value for it in the
args, block, but its associated </span><span style="color: #f1c232;">count</span><span style="font-family: inherit;"> arg has to be there, and is
type-checked normally.
</span></div>
<br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> only: does [random true]
== func [][random true]
>> blk: copy [] dup: no loop 10 [apply 'append/:only/dup [blk [1] only 2]]
== [[1] [1] 1 1 1 1 [1] [1] [1] [1] 1 1 [1] [1] [1] [1] 1 1 [1] [1]]
>> blk: copy [] dup: no loop 10 [apply 'append/:only/dup [blk [1] only none]]
*** Script Error: append does not allow none! for its count argument
*** Where: append
*** Near : apply 'append/:only/dup [blk [1] only none]
*** Stack:
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
But those aren't your only options. During the design of </span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> there was a
<i>lot</i> of discussion about the interface(s) to it, and different use cases
benefit from different models. For example, programmatically constructed calls
means you need to build the path, and keep the args in sync. It may be easier
to build a single block with everything in it, which you can do.
</span></div>
<br />
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> only: does [random true]
== func [][random true]
>> blk: copy [] loop 5 [apply :append [blk [1] /only only /dup no none]]
== [1 1 [1] [1] 1]
>> blk: copy [] loop 5 [apply :append [blk [1] /only only /dup yes 2]]
== [[1] [1] 1 1 1 1 [1] [1] 1 1]
>> blk: copy [] loop 5 [apply 'append [blk [1] /only only /dup no none]]
== [1 1 [1] [1] 1]
>> blk: copy [] loop 5 [apply 'append [blk [1] /only only /dup yes 2]]
== [[1] [1] [1] [1] 1 1 1 1 [1] [1]]
</span></code></pre>
<br />
<div style="text-align: left;"><span style="font-family: inherit;">
This interface is used if the first argument to </span><span style="color: #f1c232;">apply</span><span style="font-family: inherit;"> is a function or
lit-word, and </span><span style="color: #f1c232;">/all</span><span style="font-family: inherit;"> is not used.
</span></div>
<br />
<h3 style="text-align: left;">Apply/all</h3>
<p style="text-align: left;">
This is the most direct model, and what the others map to internally. In those
models, you get friendly refinements, which may be optional, and those may
have their own optional args. It's great for humans, and one of the best
things about Redbol languages. But look at it from the view of a function
spec.
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> print mold spec-of :append
[
{Inserts value(s) at series tail; returns series head}
series [series! bitset! port!]
value [any-type!]
/part "Limit the number of values inserted"
length [number! series!]
/only {Insert block types as single values (overrides /part)}
/dup "Duplicate the inserted values"
count [integer!]
return: [series! port! bitset!]
]</span></code></pre>
<p style="text-align: left;">
The doc string isn't used when calling functions, so we can ignore that for
this discussion. We can also ignore <span style="color: #f1c232;">return:</span> here. What's left is all the
parameters (we often interchange <span style="color: #f1c232;">arg</span> and <span style="color: #f1c232;">parameter</span>, but there's a
technical difference. Parameters are the named slots in a function spec, and
arguments are the actual values passed in those slots). There are seven of
those. Some are required args, some are refinements, and some are optional
args. But there are seven slots, and when a function is invoked it expects
there to be seven values on the stack that it <i>could</i> use if needed, or
ignored if not.
</p>
<p style="text-align: left;">
When you use <span style="color: #f1c232;">/all</span>, you're telling <span style="color: #f1c232;">apply</span> that you are going to provide
<i>all</i> those values in a single block,
<i>in the order of the function spec</i>, like a stack frame (don't worry
about the terminology too much if it's unfamiliar). <span style="color: #f1c232;">Apply/all</span> calls look
like this:
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">1.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none false true 2]]
== [1 1 1 1 1 1 1 1 1 1]
2.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none true true 2]]
== [[1] [1] [1] [1] [1] [1] [1] [1] [1] [1]]
3.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none true false 2]]
== [[1] [1] [1] [1] [1]]
4.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none only false none]]
== [1 [1] 1 1 [1]]
5.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none 1 false none]]
*** Script Error: append does not allow integer! for its only argument
*** Where: append
*** Near : apply/all 'append [blk [1] false none 1 ]
*** Stack:
6.
>> blk: copy [] loop 5 [apply/all 'append [blk [1] false none true true none]]
*** Script Error: append does not allow none! for its count argument
*** Where: append
*** Near : apply/all 'append [blk [1] false none true ]
*** Stack:
7.
>> blk: copy [] loop 5 [apply/all 'append [blk [1]]]
== [1 1 1 1 1]
</span></code></pre>
<p style="text-align: left;">
You can see that the refinement slots are now anonymous logic values in
examples 1-3, but 4 uses <span style="color: #f1c232;">only</span>, our func from earlier examples, which
randomly returns true or false. You can use anything that evaluates to
<span style="color: #f1c232;">logic!</span> for a refinement slot. 5 shows that it has to be <span style="color: #f1c232;">logic!</span>, not just
truthy, because types are checked (and logic refinement values then stand out
against none argument values). And 6 shows that if you use <span style="color: #f1c232;">/dup</span> (second from
the last arg), the <span style="color: #f1c232;">count</span> arg is also type checked, where 4 didn't complain
because <span style="color: #f1c232;">/dup</span> was false. Confused yet? Look at 7. How can that work? I
thought we had to fill all the slots! Yes and No. <span style="color: #f1c232;">Apply</span> "tail completes" the
block with false/none values for you, if you don't provide enough arguments.
Think of <span style="color: #f1c232;">find</span>, which has 16 slots. You only have to include enough args
<i>up to</i> the last one you need. That may help, but if you need to use
<span style="color: #f1c232;">/match</span>, the last refinement in <span style="color: #f1c232;">find</span>, you will have to provide all 16 args.
Before you think this is unacceptable, consider our first <span style="color: #f1c232;">repend</span> example:
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">repend: func [
{Appends a reduced value to a series and returns the series head}
series [series!]
value
/only "Appends a block value as a block"
][
append/:only series reduce :value
]</span></code></pre>
<p style="text-align: left;">It would look like this:</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">repend: func [
{Appends a reduced value to a series and returns the series head}
series [series!]
value
/as-one "Appends a block value as a block"
][
apply/all 'append [series reduce :value false none :as-one]
]
</span></code></pre>
<p style="text-align: left;">
The point here is not to show that <span style="color: #f1c232;">apply/all</span> is longer, but that we can use
a different name, where the first example must use <span style="color: #f1c232;">:only</span> in the path (make
your own version to try it). Not all refinements will propagate using the same
name. With <span style="color: #f1c232;">/all</span> it cares only about the logic value in the slot.
</p>
<h3 style="text-align: left;">Apply/safer</h3>
<p style="text-align: left;"><span style="color: #f1c232;">/Safer</span> is a form of short-circuit logic. Its purpose is to avoid the
evaluation of unused args. Without it, everything in the args block is
evaluated, but may be discarded if an associated refinement isn't active. The
easiest way to explain this is with an example.
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">; This is the function we're going to apply
applied: func [i [integer!] b /ref c1 /ref2 /ref3 c3 c4][
reduce ['i i 'b b 'ref ref 'c1 c1 'ref2 ref2 'ref3 ref3 'c3 c3 'c4 c4]
]
; And some passive and active arg values
c: 0
bar40: does [4 * 2]
baz40: does [c: c + 1 456]
; Refinement args are evaluated
apply 'applied [10 "hi" /ref on bar40 /ref3 on baz40 "ok"]
; No /safer difference because all refinement args are single values.
apply 'applied [10 "hi" /ref no bar40 /ref3 no baz40 "ok"]
apply 'applied [10 "hi" /ref no bar40 /ref3 no (c: c + 1 4 * 2) "ok"]
apply/safer 'applied [10 "hi" /ref no bar40 /ref3 no (c: c + 1 4 * 2) "ok"]
apply 'applied [10 "hi" /ref no 4 * 2 /ref3 no baz40 "ok"]
apply 'applied [10 "hi" /ref no bar40 /ref3 no c: c + 1 4 * 2 "ok"]
apply/safer 'applied [10 "hi" /ref no 4 * 2 /ref3 no baz40 "ok"]
apply/safer 'applied [10 "hi" /ref no bar40 /ref3 no c: c + 1 4 * 2 "ok"]
</span></code></pre>
<br />
<h3 style="text-align: left;">In Real Life</h3>
<p style="text-align: left;">
Here are <a href="https://github.com/red/red/commit/ca49d8d5df7d7b0bdf3e432bf571e1eb52f29dbe" target="_blank">some examples</a> of these new features being applied in the Red code
base. The <span style="color: #f1c232;">parse-trace</span> example is jaw-dropping, not because it turns 9 lines
into 1 (though, wow!), but because it makes the intent so much clearer and
eliminates so much redundant code and the errors they can lead to. Not only that, it adds a capability! Before now you couldn't use both refinements together, i.e. <span style="color: #f1c232;">parse-trace/case/part</span>, but now you can. </p>
<h3 style="text-align: left;">Things we left out</h3>
<p style="text-align: left;">
The design of <span style="color: #f1c232;">apply</span> took many turns, with long and vigorous discussion and
analysis. Many views and preferences were expressed, and which ultimately led
to dynamic refinements as what we called <b>straight sugar</b>. That is,
syntactic sugar at its sweetest. We knew <span style="color: #f1c232;">/all</span> had to be there, as that's
what the others build on, but it was originally the default. We all eventually
agreed that it shouldn't be, as it's the lowest level and likely the least
directly used, though still vital for some use cases. Our problem was striking
a balance between what would be most useful, with minimal overlap in use
cases, and making the rules too complex to remember and get right. So a couple
models didn't make the cut.
</p>
<p style="text-align: left;">
If Dynamic Refinements are straight sugar, the candy-wrapper version might be
something like this, where you can still use a path, but only the args are in
the block.
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">apply-args: function [
"Make straight sugar call from semi-sweet model."
fn [word! path!] "Get-word refinements come from context."
args [block!] "Args only, no refinement values."
][
; Use temp result so we don't return any extra args
do compose [res: (fn) (args)]
res
]
>> apply-args 'append [[] [a]]
== [a]
>> apply-args 'append/only [[] [a]]
== [[a]]
>> only: false
== false
>> apply-arg 'append/:only [[] [a]]
== [a]
>> only: true
== true
>> apply-args 'append/:only [[] [a]]
== [[a]]
</span></code></pre>
<p style="text-align: left;">
It's easy, but has quite a bit of overhead, because of <span style="color: #f1c232;">compose</span> and <span style="color: #f1c232;">do</span>.
Remember, this amount of overhead only matters in loops running thousands of
times at the very least, or in a real-time interactive interface. When in
doubt, <span style="color: #f1c232;">clock</span> it. Write for humans to understand, and only optimize as needed
(and after you know what's slow)
</p>
<p style="text-align: left;">
Another model is name-value args. That is, you provide a structure of arg
names and their values, which is applied. It can make some code much clearer,
but you also have to make sure the names match, so any refactoring of names
will break code, which doesn't happen if args are positional. This is a bit
involved, but it shows the power of Red. We'll use objects in our example, for
a particular reason. That reason is <span style="color: #f1c232;">values-of</span>. The idea being that
<span style="color: #f1c232;">apply/all</span> wants all the args, in order, and every slot filled. If your
object matches the function spec, it's a perfect match. But making objects
manually that way is error prone. So we'll use reflection for an object
tailor-made for a given function.
</p>
<p style="text-align: left;">
Step 1: Find all the words in the spec. Remember it could have doc strings and
arg types as well.
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">func-spec-words: function [
"Get all the word-type values from a func spec."
fn [any-function!]
/as type [datatype!] "Cast results to this type."
][
arg-types: make typeset! [word! lit-word! get-word! refinement!]
parse spec-of :fn [
; If we want an apply-specific version of objects, we could
; denote refinements with a sigil for added clarity.
collect [
any [set w arg-types keep (either type [to type w][w]) | skip]
]
]
]</span></code></pre>
<p>Step 2: Make an object from that</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">func-spec-to-obj-proto: function [
"Returns an object whose words match a function's spec."
fn [any-function!]
; The idea here is that you can both preset values that are in the spec,
; and also extend the object with extra words, which will be ignored.
/with args [block!] "APPLY/ALL ignores extra values."
][
obj: construct/with any [args []]
construct append func-spec-words/as :fn set-word! none
; Refinement values in APPLY/ALL calls MUST be logic!, not none!.
foreach word func-spec-words :fn [
if refinement? word [set in obj to word! word false]
]
obj
]</span></code></pre>
<p>Step Aside: Here's another approach, which combines steps 1 and 2, and lets you use a path for the function arg.</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">; Alt approach to func-spec-to-obj-proto, does NOT allow extending the spec.
make-apply-obj-proto: function [
"Returns an object whose words match a function's spec."
fn [any-function! word! path!]
/with args [block!] "TBD: If APPLY doesn't ignore extra values, keys must be in spec."
][
if path? :fn [refs: next fn fn: first fn] ; split path
if word? :fn [
name: fn ; hold on to this for error report
set/any 'fn get/any fn ; get func value
if not any-function? :fn [
do make error! rejoin ["FN argument (" name ") does not refer to a function."]
]
] ; get func value
obj: construct append func-spec-words/as :fn set-word! none ; make object
; Refinement values in apply calls MUST be logic!, not none!.
foreach word func-spec-words :fn [
if refinement? word [set in obj to word! word false]
]
if refs [foreach ref refs [obj/:ref: true]] ; set refinements
; can't use obj/:key: if key is a set-word!
if args [foreach [key val] args [set in obj key :val]] ; set arg values
obj
]
o-1: make-apply-obj-proto/with 'find/case/part/tail/skip/with [wild: "*?+" length: 10 size: 2]
</span></code></pre>
<p style="text-align: left;">Step 3. Using the object with APPLY</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">; We finally get here, and it's anticlimactic.
apply-object: func [
"Call APPLY using an object's values as args."
fn [any-function!]
obj [object!]
][
apply/all :fn values-of obj
]
</span></code></pre>
<p>And an example call:</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">>> o-fctm: make-apply-obj-proto/with 'find/case/tail/match [series: [a b c] value: 'a]
== make object! [
series: [a b c]
value: 'a
part: false
length: none
only: false
case: true
same: fal...
>> apply-object :find o-fctm
== [b c]
</span></code></pre>
<p>But you may see that this is verbose and inefficient, making a whole object just for a call like this. And you'd be right. It's just an example.<br /><br />You don't want to recreate objects like this, especially in a loop. But you don't have to. You can reuse the object and just change the important values in it. This blog is getting long already, so we'll leave that as an exercise for the reader, or a question in community chat. And if you reuse the same object, the overhead is minimal.</p>
<p style="text-align: left;">
We even talked about an idea whose time has not come, and is not guaranteed
to work in the future. Here's the idea:
</p>
<pre style="margin-bottom: 0px; margin-top: 4px; overflow: hidden;"><code style="background: rgb(35, 36, 31); border-radius: 6px; border: none; display: block; line-height: 1.4em; overflow-wrap: normal; overflow-x: auto; padding: 0.5em;"><span style="color: #f1c232;">dyna-ref: func [p [path!]][
res: make path! collect [
keep p/1
foreach val next p [
case [
get-word? val [if get :val [keep to word! val]]
all [paren? val get-word? val/1] [if do next val [keep to word! val/1]]
paren? val [do make error! "Sorry, has to be (get-word expr) for use with dyna-path."]
'else [keep val]
]
]
]
res
]
c: true
print mold dyna-ref 'a/b/:c/(:d true)
print mold dyna-ref 'a/b/:c/(:d false)
c: false
print mold dyna-ref 'a/b/:c/(:d true)
print mold dyna-ref 'a/b/:c/(:d false)
</span></code></pre>
<p style="text-align: left;">
<span>That's right, it's a dialected <span style="color: #f1c232;">path!</span> that builds a dynamic path. Crazy,
right? You may know that while paths can <i>currently</i> contain parens,
for Rebol compatibility, that feature may go away. It has deep design
implications, but is also very handy at times. And while this isn't part of
Red, it's another example of how Red lets us think off the beaten
path.</span>
</p>
<br />
<h2 style="text-align: left;">Interpreter improvements</h2>
<p>
Dynamic refinements and function application are supported at interpreter
level, for maximum efficiency and code reuse (mostly arguments fetching and
type-checking). In addition to that, long standing issues and needed
simplifications have been made in the interpreter code. Here is the changelog:
</p>
<div>
<div> Function arguments cache entirely reimplemented:</div>
<div>
<ul style="text-align: left;">
<li>
<span style="white-space: normal;">Massively reduces the amount of code needed to manage the
caches.</span>
</li>
<li>
<span style="white-space: normal;">Simpler and faster cache design, O(1) lookup time for refinements in
paths.</span>
</li>
<li>
<span style="white-space: normal;">Adds a context! to native!, action! and routine!, to speed up word
lookups.</span>
</li>
<li>
<span style="white-space: normal;">Fixes long standing issue #4854 (CRASH & CORRUPTION in "dynamic"
function calls with refinements)</span>
</li>
</ul>
</div>
</div>
<div> Simpler reimplementation of <span style="color: #f1c232;">op!</span> datatype and its evaluation:</div>
<div>
<div>
<ul style="text-align: left;">
<li><span style="white-space: normal;"><span style="color: #f1c232;">red-op!</span> structure redesigned: it is now a shallow copy of the
underlying function (nodes are not copied), the sub-type is stored in
the op! header.</span>
</li>
<li><span style="white-space: normal;"><span style="color: #f1c232;">is</span> infix function has been deprecated and replaced by a prefix
version: <span style="color: #f1c232;">relate</span>.</span>
</li>
<li>
<span style="white-space: normal;">Massive code reduction and simplification compared to previous
version. Now op! maximally reuses the interpreter code for evaluating
other functions.</span>
</li>
</ul>
<p>
Those changes lead to a general interpreter speed-up of 3-5% and up
to 20% in some cases.
</p>
</div>
</div>
<div><br /></div>
<div>Additional language changes:</div>
<div>
<ul style="text-align: left;">
<li>
The <span style="color: #f1c232;">in</span> native now accepts <span style="color: #f1c232;">any-function!</span> as its first argument and
refinements as the second argument. Refinements, if matched, will be
converted to word values. This makes <span style="color: #f1c232;">in</span> a fast way to check if a symbol
is part of an object or function spec and return a word bound to that
context.
</li>
</ul>
</div>
<br />
<h2 style="text-align: left;">Conclusion</h2>
<p>
The most important thing you should do now is try it. It's a new design,
and we want to hear what people like, see what they try, and where it falls
short.
</p>
<div>
<span><br /></span>
</div>
<div><span>Happy Reducing!</span></div><div><span><br />-Red Team</span></div>
</div>Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com8tag:blogger.com,1999:blog-5936111837781935054.post-9191504115378127402022-07-29T18:18:00.003+02:002022-07-29T18:18:54.638+02:00New Red binariesSince many years, we are offering pre-built binaries for the Red toolchain, as a more convenient way to use Red, even if it is not strictly needed, as Red can be run <a href="https://github.com/red/red#running-red-from-the-sources-for-contributors" target="_blank">from its sources</a>, the toolchain being run by a Rebol2 interpreter. As the Red REPL and toolchain are not run by the same engine, the console (REPL) used to be compiled on first run of the `<span style="font-family: courier;">red</span>` executable (when no arguments was provided or a Red script was passed). This resulted in a significant delay on the first use of the console (both for the GUI and CLI versions). <div><br /></div><div>We have now decided to change that by providing separate pre-built binaries for the consoles and toolchain. This is a temporary split until Red gets self-hosted, at which point we can recombine everything into a single binary.</div><div><br /></div><div>Another change is the temporary dropping of the semantic versioning until version 1.0 and related "stable" releases, as it seems to be too confusing to some users (Red being still in alpha stage). This also will remove a tendency from some users to care more about version increments than feature availability and work being done overall. We will now be proposing only pre-built binaries for latest commit, though older binaries will still be available if that can be of any help to anyone.</div><div><br /></div><div>So the pre-built binaries now are:</div><div><div><ul style="text-align: left;"><li>Red GUI : Red interpreter + View + GUI console</li><li>Red CLI : Red interpreter + CLI console</li><li>Red Toolchain : Encapper for Red + Red/System compiler</li></ul></div></div><div><br /></div><div>We are also considering ways to merge the GUI and CLI consoles into a single binary which can work even if no GUI API is available, falling back on CLI mode. We will also have the console(s) act as a front-end for the toolchain, even downloading it for you in the background when needed. Though for that we need a proper asynchronous `<span style="font-family: courier;">call</span>` function implementation. More news about this soon.</div><div><br /></div><div>In the meantime, enjoy running Red consoles almost instantly from just a click on the <a href="https://www.red-lang.org/p/download.html" target="_blank">Download</a> page!</div><div><br /></div><div><br /></div>Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com17tag:blogger.com,1999:blog-5936111837781935054.post-66489584514851926582022-07-14T18:41:00.000+02:002022-07-14T18:41:47.841+02:00The Road To 1.0<p>
You cannot have missed that in the last months (and even last years), our
overall progress has slowed down drastically. One of the main reasons is that
we have spread our limited resources chasing different objectives while making
little progress on the core language. That is not satisfying at all and would
bring us most likely to a dead-end as we exhaust our funding. We have spent
the last weeks discussing about how to change that. This is our updated action
plan.
</p>
<p>
From now on, our only focus will be to finish the core language and bring it
to the much-awaited version 1.0. We need to reach that point in order to
kickstart a broader adoption and provide us and our users a stable and robust
foundation upon which we can build commercial products and services necessary
for sustainability.
</p>
<p>
Given the complexities involved in completing the language and bringing an
implementation that can run on modern 64-bit platforms, we have devised a
two-stage plan.
</p>
<p><br /></p>
<h3 style="text-align: left;">Upgrade the current 32-bit Red implementation</h3>
<h4 style="text-align: left;"><br /></h4>
<h4 style="text-align: left;">👉 Language specification</h4>
<p>
It is now time to do so in order to clean-up some semantic rules and address
all possible edge cases which will help fulfill our goals of implementation
robustness and stability. The process of writing down the complete language
specs will result in dropping some features that we currently have that end up
being problematic or inconsistent. OTOH, we might add some new features that
will need to be implemented for 1.0.
</p>
<h4>👉 Modules</h4>
<p>
We need a proper module system in order to be scalable. We also need to have a
proper package management system which will be tied to a central repo where we
can gather third-party libraries. That would also enable modular/incremental
compilation (or encapping) which will be most probably supported in the
self-hosted toolchain.
</p>
<h4>👉 Concurrency</h4>
<p>
We need a proper model for concurrent execution in order to leverage multicore architectures. We will define one and make a prototype
implementation in the 32-bit version.
</p>
<h4>👉 Toolchain</h4>
<p>
Before starting to work on the new toolchain, we will make some changes to the
existing version in order to prepare for the transition. The biggest change is
the dropping of the Red compiler, which will only act as a (smart) encapper.
Routines and
<span style="font-family: courier;">#system</span> directives will still
be supported, but probably with some restrictions. The Red preprocessor might
also see some changes. This change means that Red will only have one execution
model instead of the two it has currently. The Red compiler has become more of a burden
than a help. The speed gains are not that significant in real code (even if
they can be in some micro-benchmarks), but the impossibility for the compiler
to support the exact same semantics as the interpreter is a bigger problem.
This move not only will bring more stability by eliminating some edge case
issues but also will reduce the toolchain by almost 25% in size, which will
help reduce the number of features to support in the new toolchain.
</p>
<h4>👉 Runtime library</h4>
<p>
Some improvements are long overdue in the Red runtime library. Among them:
</p>
<p></p>
<ul style="text-align: left;">
<li>unified Red evaluation stack.</li>
<li>unified <span style="font-family: courier;">node!</span> management.</li>
<li>improved processing of path calls with refinements.</li>
<li>
improved <span style="font-family: courier;">object!</span> semantics.
</li>
</ul>
<p></p>
<p>
All those changes are meant to simplify, reduce the runtime library code and
address some systemic issues (e.g. stack management issues and GC node leaks).
</p>
<h4>👉 Documentation</h4>
<p>
We need proper, exhaustive, user-oriented documentation for the Red core
language. This is one of the mandatory tasks that needs to be completed and
done well for wider adoption.
</p>
<p><br /></p>
<h3 style="text-align: left;">Self-hosted Red for 64-bit version</h3>
<h4 style="text-align: left;"><br /></h4>
<h4>👉 Toolchain</h4>
<p>
In order to go 64-bit, we have to drop entirely our current toolchain code
based on Rebol2 and rewrite it with a newer architecture in Red itself. The
current toolchain code was disposable anyway, it was not meant to live this
long, so this was a move we had to do for 1.0 anyway.
</p>
<p>So the new toolchain will feature:</p>
<p></p>
<ul style="text-align: left;">
<li>a new compilation pipeline with a plugin model.</li>
<li>
an
<a href="https://en.wikipedia.org/wiki/Intermediate_representation" target="_blank">IR</a>
layer.
</li>
<li>one or more optimizing layers.</li>
<li>modular/incremental compilation support.</li>
<li>x64, AArch64 and WASM backends.</li>
<li>linker support for 64-bit executable file formats for the big 3 OS.</li>
<li>support for linking third-party static libraries.</li>
</ul>
<div>
32-bit backends will not be supported in 1.0, though, they could be added back
in the future.
</div>
<p></p>
<h4>👉 Runtime library<br /><br /></h4>
<div>
The current Red runtime library written in R/S will be kept and some
adjustments will be needed in order to be fully compatible with a 64-bit
environment (like updating all imported OS API to their 64-bit
versions).
</div>
<div><br /></div>
<div>
View engine will not be part of that upgrade for 1.0, but will be done in a
1.1 version, priority is given to Red/Core for the 1.0.
</div>
<p><br /></p>
<h3 style="text-align: left;">Roadmap</h3>
<p>Here are the main milestones:</p>
<p></p>
<ul style="text-align: left;">
<li>v0.7 : Full I/O with async support.</li>
<li>v1.0b : (beta) completed self-hosted Red with 64-bit support.</li>
<li>
v1.0r : (release) first official stable and complete Red/Core language release.
</li>
<li>v1.1 : View 64-bit release.</li>
<li>v1.2 : Android backend and toolchain release.</li>
<li>v1.3 : Red/C3 release.</li>
<li>v1.4 : Web backend for View release.</li>
<li>v2.0 : Red JIT-compiler release.</li>
<li>v3.0 : Red/...</li>
</ul>
<p></p>
<p>
The 0.7 should be the last version for the 32-bit Red version and current
toolchain and we will be working on that first.
</p><p>For reaching the 1.0-beta milestone, we target 12 months of intensive work, so that will bring us to Q3 2023. That's an ambitious goal but necessary to reach for the sake of Red's future.</p>
<p>
The currently planned beta period for 1.0 is 2-3 months. We want a polished,
rock-solid, production-ready 1.0 release.
</p>
<p>
For the 1.1, we will probably make some (needed) improvements to View engine
architecture and backends.
</p>
<p>
For Red/C3, as the Ethereum network is transitioning to 2.0 and a new EVM, we
need the WASM backend in order to support it.
</p>
<p>
Version 1.4 will bring a proper web runtime environment to the WASM backend,
including GUI support.
</p>
<p>
The 2.0 will be focused on bringing a proper JIT-compiler to Red runtime, that
should radically improve code execution of critical parts without having to
drop to R/S.
</p>
<p>
Version 3.0 is already planned, but I will announce that once 1.0 will be
released. ;-)
</p>
<p>
One major platform is missing from the above plan, that is iOS. Given how
closed that platform is, we will need to come up with a specific plan on how
to support it, as it won't be able to cross-compile for it (you would need a
Mac computer), nor probably generate iOS apps without relying on Xcode at some
point (not even mentioning dynamic code restrictions on the AppStore), which
are layers of complexity that Red is trying to fight against in the first
place... So for now, that platform is not among our priorities.
</p>
<p><br /></p>
<p>
To finish, let me borrow some words from someone who succeeded more than
anyone else in our industry:
</p>
<div class="separator" style="clear: both; text-align: center;"><br /></div><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen="" class="BLOG_video_class" height="266" src="https://www.youtube.com/embed/0GD57UQg4X4" width="482" youtube-src-id="0GD57UQg4X4"></iframe></div><br />
<br />
<p>
Expect me to say "no" even more so from now on, as we get laser-focused on our
primary goal.<br /><br />Cheers and let's go!
</p>
Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com32tag:blogger.com,1999:blog-5936111837781935054.post-57115839539120406022021-12-31T21:23:00.004+01:002021-12-31T23:18:01.032+01:002021 Winding Down<p><br /></p>
<p><br /></p>
<p>
Another quarter, another blog post. Seems almost rushed after the previous
drought.
</p>
<p>
To set the stage, I'll start with a bit of a rant about complexity. If you
just want the meat of what's happening in the Red world, feel free to skip the
introduction.
</p>
<h2 style="text-align: left;">Complexity Considerations: Part 1</h2>
<p>
I liked what the InfoWorld article,
<a href="https://www.infoworld.com/article/3639050/complexity-is-killing-software-developers.html" target="_blank">Complexity is Killing Software Developers</a>
said, which we all know, about difficult domains (voice and image recognition,
etc.) being available as APIs. This lets us tackle things we couldn't in some
cases. Though I imagine @dockimbel or others also used Dragon Dictate's
libraries back in the 90s. What we have now is massive data to train systems
like that. Those work well, allowing us to add features we otherwise couldn't
with a small team.
</p>
<p>
The problem I see is that the trend has become for <i>everything</i> to be
outsourced, including simple features like logging, and those libraries have
exploded. There must be graphs available to show the change. Moderately
complex domains, UIs for example, have risen in number and lead to what
@hiiamboris says about Brownian Movement. It's a random collection of things,
not designed to work together, without a coherent vision. A quote from the
above article says it this way:
</p>
<p>
<span>"Complexity
is less the issue than inconsistency in an environment."</span>
</p>
<blockquote><p></p></blockquote>
<p>
It used to be that you could take a FORTRAN, COBOL, Lisp, VB, Pascal/Delphi,
Access/PowerBuilder, dBase/Clipper/Paradox, or even a Java developer, drop
them into a project, and they could work from a solid core, learning the
team's custom bits and any commercial tools as they went. With JS leading the
way, but not alone in this, a programmer can only rely on a much smaller core,
relative to how many libraries are used.
</p>
<p>
Because those libraries, and the choices to use a particular combination of
them were not designed to work together, there is no guarantee (or perhaps
hope) of consistency to leverage. It's worse if you came from a history of
other tools that were based on different principles or priorities, because you
have to unlearn, breaking the patterns in your mind. Or you convince people to
use what you did before, even if there is overlap with tools already in use.
</p>
<p>
Things are changing now, and will even more. New service-based companies are
coming, and a drive to APIs rather than libraries. So we not only have risks
like
<a href="https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/" target="_blank">LeftPad</a>, but also companies going out of business under you. The modern trend means
it's no longer dependent on an author or team committed to a project long
term, but to what investors want, and what changes are made to gain adoption
at all costs. As a service-based company you <i>can't</i> hold dearly to
design principles if the investors tell you to pivot. Because it's no longer
about your vision, but their return. If it <i>is</i> a solo FOSS author or
small team, what is their incentive to maintain a project for free, while
others profit from it? Success can be your worst enemy, and we need a more
equitable solution than what we have now. The software business model has
changed dramatically, and will likely continue to do so.
</p>
<p>
Here is what I personally see as the crux of the problem: the goal of scaling.
FOSS projects and companies are only considered successful if they have
millions (or, indirectly, billions) of users. Companies that want to be
sustainable, providing long term, moderate profits don't make headlines, but
they make the world go 'round. They are not the next big social media
disruption where end users are the product, to be bought and sold. It is a
popular business model and profit is the goal. It's nothing personal.
</p>
<p>
This has led us to the thinking that <i>every</i> project needs to be
designed for millions of users at the very least. Sub-second telemetry for all
the data collected, another explosion, giving rise to data analytics for
everyone; not just Business Intelligence (BI) for large companies. I won't
argue against having data. I love data and learning from it. But I do believe
there is a point of diminishing returns which is often ignored. Rather, in
this case, there is a cost of entry that small projects wouldn't otherwise
need to pay.
</p>
<p>
What do you do, as an "architect" (see the previous blog post about my
thoughts on software architecture) or developer on a team? Your small team (we
all know small teams are best, plenty of research and history there) simply
can't design and build every piece to support these scaling demands, while the
sword of Damocles hangs over you in the form of potential pivots (dramatic
changes in goals).
</p>
<p>
As an industry, we are being inexorably forced to make these choices. Either
you're a leader and make your own Faustian bargain, or you're in the general
mass of developers being whipped and driven to the gates of Hell.
</p>
<p>
Only you, dear reader, can decide the turns this tragic story will take, and
what you forgive in this telling perchance I should exaggerate.
</p>
<h2 style="text-align: left;">Complexity Considerations: Part 2</h2>
<p>
Complexity doesn't come only in the form
<a href="https://en.wikipedia.org/wiki/Cyclomatic_complexity" target="_blank">McCabe</a>
is famous for, the decision points in a piece of code, but in how many pieces
there are and how often they change either by choice or necessity. Temporal
Complexity if you will. This concept is unrelated to <a href="https://en.wikipedia.org/wiki/Time_complexity">algorithmic time complexity</a>. Rebol2 for any faults we can point out, still works to this day (except in
cases where the world changed out from under it, e.g., in protocols). It was
self-contained, and relied only on what the OS (Operating System) provided. As
long as OSs don't break a core set of functionality that tools rely on, things
keep working. R2 had a full GUI system (non-native, which insulated it from
changes there), and I can only smile when I run code that is 20 years old and
it works flawlessly. If that sounds silly, remember that technology, in most
cases, is <i>not</i> the goal. It is a means to an end. A lot of very old code
is still in production, keeping businesses running.
</p>
<p>
We talk about needing to keep up with changes, but some things don't change
very much, if at all. Other things change rapidly, but for no good reason, and
without being an improvement. If a change is just a lateral move there is no
value in it, unless it is to align us on a different, and better, path in the
future. I started programming with QuickBASIC, but also used other tools as I
quickly learned my tool of choice came with a stigma attached, and I wanted to
be a serious, "real" programmer. What became clear was that QB was a great
tool, with a few companies providing terrific ASM libraries, and had a
wonderful IDE to boot. It was simpler, not only as a language, but because
every 12-18 months (the release cycle way back when) my new C compiler would
break something in my code. But QB, and later BASIC/PDS and then VB very
rarely broke working code. Temporal complexity.
</p>
<p>
Even then there were more complex options. The cool kids used Zortech C++ and
there were various cross-platform GUI toolkits. But those advanced tools were
often misapplied to simple projects. We still do that today. Much of that is
human nature, and the nature of programmers. If it's easy we are no longer
special. We may not mean to, but we make things harder than they need to be.
Some of us are even elitist about what we do, to our own detriment. If you
don't need to be cross platform, why do you have multiple machines or VMs each
with a different compiler setup? If you need a GUI, why are you using a
language that was not designed with them in mind? If you need easy deployment,
which is simpler: a single EXE with no dependencies, or a containerization
approach with all that entails? How many technologies do you <i>need</i> in
your web stack? Are you the victim of peer pressure, where you feel your site
has to be shiny and "responsive", or use the latest framework?
</p>
<p>
A big argument for using other's work is performance. They've taken time, and
may be experts, to optimize <span style="color: #6aa84f;">Thing X</span> far
beyond what you could ever do. That JIT compiler, an incredible virtual DOM,
such clever CSS tricks, the key-value DB with no limits, and yet...and yet our
software is slower and more bloated than ever. How can that be? Is it possible
we're overbuilding? Is software sprawl just something we accept now?
</p>
<p>
Earlier I mentioned that a hodge-podge assembly of parts that have no
standards, norms, or even aesthetic sense applied does not make our lives
easier. Lego blocks, the originals anyway, are limited, but consistent in how
they can be used. We misapply that analogy, because the things we build are
far from consistent or designed to interact. Even in the realm of UX and A/B
testing on subsets of users that companies apply today. I <i>love</i> the
idea of data-driven HCI to guide us to a more evidence-oriented approach. This
includes languages. But when a site or service moves fast and changes their
interface based on their own A/B testing, they don't account for the others
doing the same. Temporal complexity.
</p>
<p>
As a user, every app or site I access may change out from under me in the
flash of refresh or automatic update I didn't ask for. Maybe it's better, an
actual improvement, if you only use that one site. But if all your tools
constantly change out from under you, it's like someone sneaking into your
office and rearranging it every night while you sleep. Maybe this is the
developer's revenge, for the pain we inflict on ourselves by constantly
changing our own tools. If we suffer, why shouldn't our users? For those who
truly have empathy for their users and don't want to drive them mad, or away,
perhaps the lesson is to have empathy for ourselves, for our own tribe. I
don't want to see my friends and colleagues burn out, when it was probably the
enjoyment and passion that solving problems with software can bring which led
them here to begin with.
</p>
<p>
Every moving part in your system is a potential point of failure. Reduce the
moving parts and reliability increases. Whether it's the OS you run on (we now
have more of those than ever, between Linux distros and mobile platforms
always trying to outdo each other), extra packages or commercial tools, FOSS
libraries, environments, <span style="color: #6aa84f;">[?]aaS</span>, or
platform components like containers and cluster management, every single piece
is a point of failure. And if any of them break your code, or your system,
even in the name of improvements or bug fixes, you may find yourself running
just to stay in the same place. Many of those pieces are touted as the
solution to reliability problems, but a lot of them just push problems around,
or target problems you don't have. Don't solve problems you don't have. That
adds complexity, and now you really have a problem.
</p>
<h2 style="text-align: left;">Less Philosophy, More Red</h2>
<div><br /></div>
<h3 style="text-align: left;">Interpreter Events</h3>
<div><br /></div>
<div>
Having a debugger in Red has been a request of many users for a long time,
even since the Rebol era. We have tackled this feature from a larger
perspective, considering general instrumentation of the interpreter (note: not
the compiler), extending it with an event system and user-provided event
handlers, similar to how
<span style="color: #6aa84f; white-space: pre-wrap;">parse</span> and
<span style="color: #6aa84f; white-space: pre-wrap;">lexer</span>
tracing operate today. This approach allows us to build more than just a
debugger, though it was a lot of work to design and we expect it will be
refined once people start using it in earnest. It's a brave new world, with a
lot of tooling possibilities.
</div>
<div><br /></div>
<div>
It's important to note that this is not magic. Because it operates as the
interpreter evaluates values and expressions, including functions, it can't
see into the future. In order to get a complete trace, you have to evaluate
everything. That means we'll see tools which silently collect data, like a
profiler does, which can later be viewed and analyzed, perhaps up to the point
where an error occurred. This is an important aspect, and plays once again
into the power of Red as data. Your event handlers can easily collect data
into any structure or model you like. And because event handlers can filter
events, you can tailor them for specific needs. It should even be possible to
build interpreter level
<a href="https://en.wikipedia.org/wiki/DTrace" target="_blank">DTrace</a>-like
tools in the future. We also hope to build higher level observability and
monitoring tools, based on eventing systems, in the future, but those are long
term projects.
</div>
<div><br /></div>
<div>Event generation is not active by default, it is enabled using
<span style="color: #6aa84f; font-family: courier;">do/trace</span> and by
providing an event handler function. For example, here's a simple logging
function:
</div>
<div>
<pre class="code"> logger: function [
event [word!] ;-- Event name
code [any-block! none!] ;-- Currently evaluated block
offset [integer!] ;-- Offset in evaluated block
value [any-type!] ;-- Value currently processed
ref [any-type!] ;-- Reference of current call
frame [pair!] ;-- Stack frame start/top positions
][
print [
pad uppercase form event 8
mold/part/flat either any-function? :value [:ref][:value] 20
]
]
</pre>
Given this code:
<pre class="code"> do/trace [print 1 + 2] :logger</pre>
It will output:
<pre class="code"> INIT none ;-- Initializing tracing mode
ENTER none ;-- Entering block to evaluate
FETCH print ;-- Fetching and evaluating `print` value
OPEN print ;-- Results in opening a new call stack frame
FETCH + ;-- Fetching and evaluating `+` infix operator
OPEN + ;-- Results in opening a new call stack frame
FETCH 1 ;-- Fetching left operand `1`
PUSH 1 ;-- Pushing integer! value `1` on stack
FETCH 2 ;-- Fetching and evaluating right operand
PUSH 2 ;-- Pushing integer! value `2`
CALL + ;-- Calling `+` operator
RETURN 3 ;-- Returning the resulting value
CALL print ;-- Calling `print`
3 ;-- Outputting 3
RETURN unset ;-- Returning the resulting value
EXIT none ;-- Exiting evaluated block
END none ;-- Ending tracing mode
</pre>
Several tools are now provided in the Red runtime library, built on top of
this event system:
<div>
<ul style="text-align: left;">
<li>
An <a href="https://github.com/red/docs/blob/master/en/interp-events.adoc#debug" target="_blank">interactive debugger</a> console, with many capabilities (step by step evaluation, a flexible breakpoint system, and call stack visualisation).
</li>
<li>
A simple <a href="https://github.com/red/docs/blob/master/en/interp-events.adoc#profile" target="_blank">profiler</a> that we will improve over time (especially on the accuracy aspects).
</li>
<li>
<div>
A simple <a href="https://github.com/red/docs/blob/master/en/interp-events.adoc#trace-1" target="_blank">tracer</a>. The current evaluation steps are quite low-level, but @hiiamboris has already built <a href="https://gitlab.com/hiiamboris/red-mezz-warehouse/-/blob/master/tracing.md" target="_blank">an extended version</a>, operating at the expression level that will soon be integrated into the master branch.
</div>
</li>
</ul>
</div>
<div>
Full docs are <a href="https://github.com/red/docs/blob/master/en/interp-events.adoc" target="_blank">here</a>.
</div>
<div><br /><br /></div>
</div>
<h3 style="text-align: left;">Format</h3>
<div><br /></div>
<div>
Boy, I really thought this was was going to be easy, or at least not
<i>too</i> hard. I couldn't have been more wrong. When I did my format
experiments, I imagined at least some of the code would be useful, requiring
polish and more work of course, providing a foundation to work from. It turns
out that I missed a key aspect, and my approach was just one of many possible.
@hiiamboris and @giesse both weighed in, and we chatted about specific parts.
Then it sat idle for a while, and I asked Boris to take it over to get it into
production. He identified the key missing piece, which would have limited its
usefulness until we eventually had to address it. Better now than later. He
also made a strong case for a different approach to the core masked-number and
I told him to run with it. That led to a lot of design chat about one aspect,
which is as yet undecided. It's not a fight to the death, but there has
definitely been some sparring. :^)
</div>
<div><br /></div>
<div>
The missing piece I've alluded to is Localization (L10N). As an American who
has never had to develop software requiring Internationalization (I18N), I've
been blissfully ignorant of all the aspects that come into play when
Globalization (G11N) becomes part of the process. We have talked about how to
implement L10N in Red, and have
<span style="color: #6aa84f; font-family: courier;">system/locale</span> for a
months, weekdays, and currency codes. The first two we inherited from Rebol's
design, the latter was added when @9214 designed the
<span style="color: #6aa84f; font-family: courier;">currency!</span> datatype.
Thinking of locale data in a system catalog of some kind is easy enough, but
how to actually apply it (and not apply it when necessary) is a different
story entirely. And I mean <i>entirely</i>. Format forced us to start down
this path, and is a guinea pig feature that will guide future plans for all
future L10N work. But keep my complexity rant in mind. While we want to make
it as easy as possible for Reducers to write globally aware apps, if you don't
need it, don't do it. We don't yet know if we can make it so magical that you
can write your app ignoring that for the most part, and then flip a switch, or
simply include local data, and have it work. Don't get your hopes up. There's
a lot that can go wrong with that approach.
</div>
<div><br /></div>
<div>
We agreed to start with masked numbers but, in order to do that, L10N R&D
had to be done. This led to broad and deep dives into unicode.org and other
resources. While they cover far more than we need, and is overly complex in
many cases (or just doesn't match our aesthetic sense for Red), the data they
have there is enormously valuable, and we deeply appreciate it being
available. We just draw the line around a smaller scope than they do, and no
committees are involved where people fight to get their own bits included.
Well, we do that too, to some extent. What Boris managed to do was identify
the key elements needed for our work, and then wrote tools (using Red of
course) to extract and reformat the data for use in Red. I can't stress how
much work this was. Truly a heroic and mostly thankless effort most people
will never know about.
</div>
<div><br /></div>
<div>
In order to test masked number formatting, and give others an easy way to
play, Boris created a <a href="https://files.gitter.im/5565a1d415522ed4b3e10094/r8VT/playground-signed.exe">Playground App</a> and I can't tell you how important that was. You see, a particular
piece of behavior came up while I was playing with it and got unexpected
results. Unexpected to me, but Boris confirmed it was by design. I will just
say here that it's about a significant digits mode, and let you play with the
app from there. Named formats will be available, but everything will likely
boil down to wrappers around masks, which should cover almost any need.
</div>
<div><br /></div>
<div>
Next up is date formatting. This time I knew locales would play a role because
some IETF RFCs specify that date elements be in English. So you may have
localized dates for some things, but if you use RFC2822 dates or HTTP cookie
dates, they <i>must not</i> be affected by any locale settings. Dates will use
masks at the core, like numbers, because masks are an easy to understand
WYSIWYG format. Well, easy if the masks make sense. If you look at
<span style="color: #6aa84f; font-family: courier;">printf</span> and some
other mask syntax, it can be quite obscure. By trying to cram things into a
limited syntax, people end up using whatever low ASCII letters might be left over
for some elements. We hope to avoid that.
</div>
<div><br /></div>
<div>
<span style="font-family: inherit;">Our main choices are what Boris termed the stuttering format. e.g.
MMDDYYYY/HHMMSS. Think in terms of "<span>progressing in a hesitant or irregular way."</span><span>
rather than stuttering in terms of human speech. I prefer to call this a
<span style="color: #6aa84f;">symbolic</span><span>
format, where the letters map to date elements. This, of course, isn't
perfect. e.g. is MM month or minute? Context is required. We don't want
to be case sensitive, or use other letters randomly to avoid that
conflict. So there's an alternate approach; a </span><span style="color: #6aa84f;">literal</span><span>
mask. e.g. 1-Jan-2022. We're not the first to consider it, and it is in
use elsewhere, but it's not a perfect solution either. Do masks have to
be written in English terms, or can they use any locale? How do you
disambiguate numbers (does 01-01 mean MM-DD or DD-MM, and how do you
write that without the separator to get MMDD?) Does it make code more or
less readable, because Red already has a literal date form, and it would
add what look like literal dates as strings in code.</span></span></span>
</div>
<div>
<span><br /></span>
</div>
<div>
Play with the app, give us feedback, and stay tuned. We think this will be a
crucial feature for a lot of users, and we want to make it the best it can be.
</div>
<div><br /></div>
<h3 style="text-align: left;">Split</h3>
<div><br /></div>
<div>
Like <span style="color: #6aa84f; font-family: courier;">format</span>,
<span style="color: #6aa84f; font-family: courier;">split</span> seems a relatively simple subject at a glance. And if you limit it to basic
functionality, it is. That's what other languages do, though some add a few
extra features. See
<a href="https://github.com/greggirwin/red-split/blob/master/split-in-other-languages.md">this table</a>
for examples. Wolfram <i>appears</i> quite broad in scope, because there are
multiple variants for each named function. Something else common to all other
languages is that they split only strings and sometimes byte arrays. In Red we
have blocks, and while `parse` is great for string parsing, where it really
shines is when applied to blocks to build dialects. We knew <span style="color: #6aa84f; font-family: courier;">split</span> should be
block aware for more leverage. I (Gregg) helped design the version in R3, and
used
<a href="https://www.redlake-tech.com/products/diagrammar-for-windows/">DiaGrammar</a>
to design a new dialected interface that aimed to extend the functionality.
Wanting to do more evidence based language design, I also prototyped a small
practice/playground app, thinking we'd put it out and see what kind of
feedback we could get.
</div>
<div><br /></div>
<div>
Toomas stepped up and suggested an alternative, refinement-based, interface.
He did a number of versions of that, and then we had to decide what to do
next. There was a great deal of design discussion, still going on, about
behavior details. Once you start adding options, it's easy for things to
become confusing for the user. We need to strike a balance between ease of use
and flexibility.
<span style="color: #6aa84f; font-family: courier;">Split</span> is meant
to handle the most common cases, and those with the most leverage, not
<i>every</i> case. And while a refinement-based interface seems natural for
Reducers, we also know how readable
<a href="https://github.com/red/docs/blob/master/en/parse.adoc" target="_blank">parse</a>,
<a href="https://github.com/red/docs/blob/master/en/draw.adoc" target="_blank">draw</a>, and
<a href="https://github.com/red/docs/blob/master/en/vid.adoc" target="_blank">VID</a>
dialects are. There are pros and cons to each, but we don't want dual
interfaces, which will be confusing. If a function is dialected, any
refinements should work in support of that dialect. So the test app was
reimagined by @GalenIvanov to compare the two approaches.
</div>
<div><br /></div>
<div>
Here's a screenshot of the test app, which we'll release to the community in
January.
</div>
<div><br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/a/AVvXsEh8adjrXvAefvFESPdf33uk9UwaBwH5yVJ5IgXH8mj7k80H18t2oTOGJ-p9Lt_MJ64BNRQu6b5BAruex_MlVxYThBVXefYmlApk9c7jONbp1QxbamKR5f6khmaH7kAnqX1JFBl-HNzzrLN4J75K8OD8CiHaa2ilVbcJocj380vTXwEZ3FoE8moeUvSq=s817" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="617" data-original-width="817" height="361" src="https://blogger.googleusercontent.com/img/a/AVvXsEh8adjrXvAefvFESPdf33uk9UwaBwH5yVJ5IgXH8mj7k80H18t2oTOGJ-p9Lt_MJ64BNRQu6b5BAruex_MlVxYThBVXefYmlApk9c7jONbp1QxbamKR5f6khmaH7kAnqX1JFBl-HNzzrLN4J75K8OD8CiHaa2ilVbcJocj380vTXwEZ3FoE8moeUvSq=w479-h361" width="479" /></a>
</div>
<br />
<div>
We learned by doing this that it's hard to compare them side by side, without
having the user write full calls directly. That defeats some of the purpose,
and the DRY principle, so we'll put this one out, then revise it based on
feedback.
</div>
<div><br /></div>
<h3 style="text-align: left;">Markup Codec</h3>
<div><br /></div>
<div>
Who knew that parsing HTML and XML would be the easy part? Well, many Reducers
would. What they, and we on the team, might not have guessed, is just how hard
it is to decide on a data format for the output. Red gives us many options,
and XML gives us many headaches. The two formats, while closely related, also
have some critical differences. Fortunately, once @rebolek set things up so we
could play, and made the emitter modular, we could look at real examples and
dive even deeper. What we discovered is that there is no perfect solution. No
elegant model to fit all uses and cases. Key to many insights was @dander's
input, as he works with XML a lot. Turns out, an infinitely extensible format
is infinitely challenging to nail down.
</div>
<div><br /></div>
<div>
Should we emphasize path access? Being data driven, people probably
<i>shouldn't</i> hardcode their field names, but working with known data makes
it a clear access model. Should attributes come before or after the
text/content for a tag? As we learned, attributes aren't always small, so the
locality argument isn't won either way there. Is it better to provide an
interface to the structure and tell people to always use that, or to create a
bland and obvious data structure that is possible to access in many ways? Will
these things all complicate HOF access, which we know we want to leverage? How
much do we need to care about efficiency? We don't want to be wasteful without
purpose, but if we're too miserly, users may pay the price because it's harder
to use. If we make more things implicit, do we paint ourselves into a corner
somehow?
</div>
<div><br /></div>
<div>
What we settled on was a modular approach, so there will be more than one
standard emitter. What is yet undecided is how other emitters might be
supported. They will likely be quite custom, as the standard versions will
cover most needs. But is it worth making the system extensible? Once you have
a result, it's easy to post-process into your preferred format. For now that's
our recommended approach.
</div>
<div><br /></div>
<h3 style="text-align: left;">CLI Module</h3>
<div><br /></div>
<div>
If you don't follow our channels on Gitter, you may not know about
<a href="https://github.com/hiiamboris/red-cli-gitlab-mirror" target="_blank">Boris' CLI module</a>. It's very slick, very Reddish, and will become a standard part of Red in
the near future. You won't believe how easy it is to create rich command line
interfaces for your Red apps with this feature. Huge thanks to @hiiamboris for
all his innovation and work on it.
</div>
<div><br /></div>
<h3 style="text-align: left;">IPv6 Datatype</h3>
<div><br /></div>
<div>
It hasn't been merged to the mainline yet, but it's fully operational. You can
see the code
<a href="https://github.com/red/red/blob/IO/runtime/datatypes/ipv6.reds" target="_blank">here</a>, and some lexer tests
<a href="https://github.com/red/red/blob/IO/tests/source/units/lexer-test.red#L415" target="_blank">here</a>. You may be impressed that it's only a couple hundred lines of code, not
counting the lexer changes, and think it was easy. It wasn't. As usual, there
was a lot of design chat and compromise involved. For example, the name is not
100% finalized because, technically, the datatype itself is more generally
applicable, being simply a vector of numbers internally. You can think of it
like a <span style="color: #6aa84f; font-family: courier;">tuple!</span> on
steroids. Less slots (8 vs 12), but each slot can hold a larger value (<span style="color: #6aa84f; font-family: courier;">tuple!</span>
slots are limited to byte values).
</div>
<div><br /></div>
<div>
Just as <span style="color: #6aa84f; font-family: courier;">tuple!</span> is a general name, used both for IPv4 addresses and colors, but also
useful for other things, <span style="color: #6aa84f; font-family: courier;">IPv6!</span> could be used for things like GUIDs or extended time values. But the
lexical form for GUID/UUID values is quite different, even ignoring the
shortcut forms in the
<a href="https://datatracker.ietf.org/doc/html/rfc4291" target="_blank">IPv6 specification</a>. As you probably know, lexical space is tight in Red, and the colon is an
important character in other places, and URL lexical forms were impacted, so
this is a deep change and commitment, in that regard. Why do it then?
</div>
<div><br /></div>
<div>
Because IPv6 networking support was already in place in Red, and IPv6 is the
future. How often people will write literal URLs like
<span style="color: #6aa84f; font-family: courier;">http://[FEDC:BA98:7654:3210:FEDC:BA98:7654:3210]:80/index.html</span>
we can't say. But we do know that addresses often end up in config files as
data and that modern, dynamic systems generate addresses dynamically. They
will appear in log files, messages, and more. As with the value of other
lexical forms in Red, it's an important one that is part of our modern
networking vocabulary.
</div>
<div><br /></div>
<h3 style="text-align: left;">Getting Near</h3>
<div><br /></div>
<div>
@dockimbel created a new branch
<a href="https://github.com/red/red/tree/error-near" target="_blank">here</a>,
which will interest almost every Reducer. It's not ready yet, but expect it to
be available in January. For those who used R2, you may recall that errors
gave you a Near field, to hint at where the error occurred. Red will get this
feature when the new branch is merged. e.g., in Red today you get this:
</div>
<div><br /></div>
<div>
<div> >> 1 / 0</div>
<div> *** Math Error: attempt to divide by zero</div>
<div> *** Where: /</div>
<div> *** Stack: </div>
</div>
<div><br /></div>
<div>Where in R2 you got this:</div>
<div><br /></div>
<div>
<div> >> 1 / 0</div>
<div> ** Math Error: Attempt to divide by zero</div>
<div> ** Near: 1 / 0</div>
</div>
<div><br /></div>
<div>
A little extra information goes a long way. We're anxious to see all the
virtual smiles this features brings.
</div>
<div><br /></div>
<h2 style="text-align: left;">The Daily Grind</h2>
<div><br /></div>
<div>
We closed roughly 120 tickets in 2021, that's 10 per month. We also merged
almost 50 PRs. These numbers don't sound large, but when you consider how much
time and effort may go into the deep ones, along with all the other work done,
it's steady progress. We'd love for both tickets and pending PRs to be at
zero, but that's not practical for a project like Red. The deep core team must
have uninterrupted time for design and bigger, more complex tasks.
</div>
<div><br /></div>
<h2 style="text-align: left;">Roadmap</h2>
<div><br /></div>
<h3 style="text-align: left;">Q4 2021 (retrospective)</h3>
<div><br /></div>
<div>
<ul style="text-align: left;">
<li>
We hoped to have `format` and `split` deployed, but they will push back to
Jan-2022.
</li>
<li>
`CLI` module approved, needs to be merged, then refined as necessary.
</li>
<li>
`Markup Codec` took longer than expected due to extensive design chat on
formats.
</li>
<li>
Interpreter instrumentation, with PoC debugger and profiler. Took longer
than expected, but are out now.
</li>
<li>
Async I/O, out but some extra bits didn't make it in. One unplanned
addition was `IPv6!` as a datatype. It's experimental, and subject to
change.
</li>
<li>
@galenivanov did some great work on his animation dialect, but @toomasv's
`diagram` dialect took a back seat and will move to Q1 2022.
</li>
<li>
Audio has 3 working back ends and a basic port implementation. Next up is
higher level design, device and format enumeration, and device control. A
`port!` may not be the way to go for all this, but it was step one.
</li>
<li>
Animation has more great examples all the time. Like
<a href="https://files.gitter.im/611aa1b26da03739848349bc/ppfZ/Anim-text-fx.gif" target="_blank">this</a>
and
<a href="https://files.gitter.im/611aa1b26da03739848349bc/d4Pc/Animate-markers.gif" target="_blank">this</a>. @GalenIvanov is doing great work, and we are planning to make his
dialect a standard addition to Red.
</li>
</ul>
</div>
<h3 style="text-align: left;"><br /></h3>
<h3 style="text-align: left;">2022</h3>
<div><br /></div>
<div>
I'm not going to list items in any particular order, because our plans often
change. This way you have things to look forward to, but still with an element
of surprise.
</div>
<p></p>
<ul style="text-align: left;">
<li>`Table` module, `node!` datatype and other REP reviews</li>
<li>Full HTTP/S protocol and basic web server framework</li>
<li>
New
<a href="https://www.redlake-tech.com/products/diagrammar-for-windows/" target="_blank">DiaGrammar</a>
release
</li>
<li>Animation dialect</li>
<li>New release process</li>
<li>New web sites updated and live</li>
<li>Red/C3 (Including ETH 2.0 client protocol)</li>
<li>
Red Language Specification (Principles, Core Language, Evaluation Rules,
Datatype Specs (including literal forms), Action/Native specs, Modules spec.
</li>
<li>
64-bit support (LLVM was a possibility, but we learned from Zig that LLVM
breaking changes can be quite painful for small teams to keep up with. We
may be better off continuing to roll our own, though it's a big task.)
</li>
<li>Android update</li>
<li>Red Spaces cross-platform GUI</li>
<li>Module and package system design</li>
<li>RAPIDE (Rapid API Development Environment)</li>
</ul>
<div><br /></div>
<h2 style="text-align: left;">RAPIDE, from Redlake Technologies</h2>
<div><br /></div>
<div>
If you've used
<a href="https://www.postman.com/" target="_blank">Postman</a> or
<a href="https://insomnia.rest/" target="_blank">Insomnia</a>, you know what
the most popular tools in the API IDE space look like today. If you haven't
used them, but use APIs, they're worth a look. For all that those tools do,
and there are a few other players in the space, there is a lot they don't do.
We think we can add a lot of value in the API arena, thanks to Red's
superpowers and how important data-centric thinking is. For example, testing a
group or series of APIs together seems like it could be greatly improved.
Also, how APIs are found, and collaboration possibilities.
</div>
<div><br /></div>
<div>
While we haven't set a release date, the plan is to start work on RAPIDE in Q2
2022, after we wrap up some infrastructure pieces it will rely on.
</div><div><br /></div><div><br /></div><h2 style="text-align: left;">In conclusion</h2><div><br /></div><div>Happy New Year to all, and may 2022 see us all healthy, happy, and writing more Red. :^)</div>
<p></p>
Unknownnoreply@blogger.com11tag:blogger.com,1999:blog-5936111837781935054.post-73422502420491220822021-08-04T02:21:00.000+02:002021-08-04T02:21:31.734+02:00Long Time No Blog<p> It's been almost a year since our last blog post. Sorry about that. It's one of those things that falls off our radar without a person dedicated to it, and we run lean so don't have anyone filling that role right now. We know it's important, even if we have many other channels where people can get information. So here we are.</p><p>Last year was a tough year all around, even for us. We were already a remote-only team, but the effect the pandemic had on the world, particularly travel, hit us too. We had some team changes, and also split our focus into product development alongside core Red Language development. This is necessary for sustainability, because people don't pay for programming languages, and they don't pay for Open Source software. There's no need to comment on the exceptions to these cases, because they are exceptions. The commercial goal, starting out, is to focus on our core strengths and knowledge, building developer-centric tools. Our first product, <a href="https://www.redlake-tech.com/products/diagrammar-for-windows/" target="_blank">DiaGrammar for Windows</a>, was released in December 2020, and we've issued a number of updates to it since then. Our thanks to Toomasv for his ingenuity and dedication in creating DiaGrammar. We are a team, but he really accepted ownership of the project and took it from an idea to a great product. Truly, there is nothing else like it on the market. </p><p>We learned a lot from the process of creating a product, and will apply that experience moving forward. An important lesson is that the product itself is only half the work. As technologists, we're used to writing the code and <i>maybe</i> writing some docs to go with it. We don't think about outreach, marketing, payments, support, upgrade processes for users, web site issues, announcements, and more. The first time you do something is the hardest, and we're excited to improve and learn more as we update DiaGrammar and work on our next product. We'll probably announce what it will be in Q4. One thing we can say right now is that the work on DiaGrammar led to a huge amount of work on a more general diagramming subsystem for Red. It's really exciting, and we'll talk more about that in a future blog post.</p><h2 style="text-align: left;">So what <i>have</i> we been doing?</h2><p>Since our last blog post we've logged over 400 fixes and 100 features into Red itself. Some of these are small, but important, others are headline-worthy; some are deep voodoo and some visible to every Reducer (what we call Red users). For example, most people use the console (the REPL), so the fixes and improvements there are easy to see. A prime feature being that the GUI console, but not the CLI console, didn't show output if the UI couldn't process events. This could happen if you printed output in a tight loop. The results would only show up at the termination of the loop, when the system could breathe again. That's been addressed, but wasn't easy and still isn't perfect. Red is still single threaded, so there's no separate UI thread (pros and cons there). We make these tradeoffs every day, and need feedback from users and real world scenarios to help find the right balance. Less obvious are things like improvements to <span style="color: #6aa84f; font-family: courier;"><b>parse</b></span>, which not everyone uses. Or how <span style="color: #6aa84f; font-family: courier;"><b>fmod</b></span> works across platforms, and edge cases for lexical forms (e.g. is <span style="color: #6aa84f; font-family: courier;"><b>-1.#NaN</b></span> valid?). The latter is particularly important, because Red is a data language first.</p><p>JSON is widely used, but people may not notice that the JSON decoder is 20x faster now, unless they're dealing with extremely large JSON datasets. JSON is <i>so</i> widely used that we felt the time spent, and the tradeoffs made, were worth it. It also nicely shows one of Red's strengths. Profiling showed that the codec spent a lot of time in its <span style="color: #6aa84f; font-family: courier;"><b>unescape</b></span> function. @hiiamboris rewrote that as a Red/System routine, tweaked it, and got a massive speedup. No external compiler needed, no need to use C, and the code is inlined so it's all in context. Should your JSON be malformed, you'll also get nicer error information now. As always, Red gives you options. Use high level Red as much as possible, for the most concise and flexible code, but drop into Red/System when it makes sense.</p><p>Some features cross the boundary of what's visible. A huge amount of work went into D2D support on Windows. D2D is Direct2D, the hardware-accelerated backend for vector graphics on Windows. For users, nothing should change as all the details are hidden. But the rendering behavior is not exactly the same. We try to work around that, but sometimes users have to make adjustments as well; we know because DiaGrammar is written in Red and uses the <span style="color: #6aa84f; font-family: courier;"><b>draw</b></span> dialect heavily. It's an important step forward, but comes at a cost. GDI+ is now a legacy graphics back end, and won't see regular updates. Time marches on and we need to look forward. As if @qtxie wasn't busy enough with that, he and @dockimbel also pushed Full I/O forward in a big way. It hasn't been merged into the main branch yet, but we expect that to happen soon. @rebolek has been banging on it, and has a full working HTTP protocol ready to go, which is great. TLS/SSL support gets an A+ rating, which is also a testament to the design and implementation. It's important to note that the new I/O system is a low level interface. The higher level API is still being designed. At the highest level, these details will all be hidden from users. You'll continue to use <span style="color: #6aa84f; font-family: courier;"><b>read, write, save, load</b></span> exactly as you do today, unless you need async I/O. </p><p>Another big "feature" came from @vazub: NetBSD support. The core team has to focus on what stands to help the project overall, with regard to users and visibility. Community support for lesser known platforms is key. If you're on one of those platforms, be (or find) a champion. We'll help all we can, but that's what Open Source is for. Thanks for this contribution @vazub!</p><p>We also have some new Python primers up, thanks to @GalenIvanov. Start at <a href="https://github.com/red/red/wiki/Coming-to-Red-from-Python" target="_blank">Coming-to-Red-from-Python</a>. Information like this is enormously important. Red is quite different from other languages, and learning any new language can be hard. We're used to a set of functionality and behaviors, which sometimes makes the syntax the easiest part to learn. Just knowing what things are called is a learning curve. Red doesn't use the same names, because we (and Carl when he designed Rebol) took a more holistic view. That's a hard sell though. We feel the pain. A user who found Red posted a video as they tried to do some basic things. We learned a lot from watching it. Where other languages required you to import a networking library, it's already built into Red. When they were looking for <span style="color: #6aa84f; font-family: courier;"><b>request</b></span> or <span style="color: #6aa84f; font-family: courier;"><b>http.get</b></span>, and expecting strings to be used for URLs, they couldn't find answers. In Red you just <span style="color: #6aa84f; font-family: courier;"><b>read http://...</b></span>. It's obvious to us, but not to the rest of the world. So these new primers are very exciting. We have reference docs, and <a href="https://www.red-by-example.org/">Red by Example</a>, but still haven't written a User's Guide for Red. We'll get there though. </p><h2 style="text-align: left;">Why do things take so long?</h2><p>Even with that many fixes and features logged, and huge amounts of R&D, it can still feel like progress is slow. The world moves fast, and software projects are often judged by their velocity. We even judge ourselves that way, and have to be reminded to stay the course, our course, rather than imitating others. Red's flexibility also comes into play. Where other languages may limit how you can express solutions, we don't. It's so flexible that people can do crazy things or perform advanced tricks which end up being logged as bugs and wishes. Sometimes we say No (a lot of times in fact), but we also try to keep an open mind. We have to ask "<i>Should</i> that be allowed?", "<i>Why</i> would you want to do that (even though I never have)?", and "What are the long term <i>consequences</i>?" We have to acknowledge that Red is a data format first, and we never want to break that. It has to evolve, but not breaking the format is fundamental. And while code is expected to change, once people depend on a function or library it causes them pain if we break compatibility. We don't want to do that, though sometimes we will for the greater good and the long view. There are technical bandages we can patch over things, but it's a big issue that doesn't have a single solution. Not just for us, but for all software development. We'll talk more about this in the future as well.</p><p>I'll note some internal projects related to our "slow and steady" process:</p><p></p><ul style="text-align: left;"><li><a href="https://gitlab.com/hiiamboris/red-mezz-warehouse/-/blob/master/composite.md" target="_blank">Composite</a> is a simple function that does for strings what <span style="color: #6aa84f; font-family: courier;"><b>compose</b></span> does for blocks. It's a basic interpolator. But the design has taken many turns. Not just in the possible notations, but whether it should be a mezzanine function, a macro, or both. Each has pros and cons (Side note: we don't often think about "cons" being an abbreviation for "consequences"). This simple design and discussion is stalled again, because another option would be a new literal form for interpolated strings. That's what other languages do, but is it a good fit for Red? We belabor the point of how tight lexical space is already, so have to weigh that against the value of a concise notation.</li><li>Non-native GUI. Red's native GUI system was chosen in response to Rebol's choice to go non-native. Unfortunately it's another case of needing both. Being cross platform is great for Red users, but Hell for us. Throw in mobile and it's even worse. Don't even talk about running in the browser. But <i>every</i> platform has native widget limitations. Once you move beyond static text, editable fields, buttons, and simple lists, you're in the realm of "never the twain shall meet". How do you define and interact with grids and tables or collapsible trees? Red already has its own <a href="https://github.com/red/docs/blob/master/en/rtd.adoc" target="_blank">rich-text</a> widget, so you don't have to embed (even if you could) an entire web browser and then write in HTML and CSS. To address all this, with much research and extensive use case outlines, @hiiamboris has spent a lot of time and effort on <a href="https://gitlab.com/hiiamboris/red-spaces" target="_blank">Red Spaces</a>. Show me native widgets that can do editable spiral text, put any layout inside a rotator, or define recursive UIs. I didn't think so. Oh, and the wiggling you see in the GIFs there are not mistakes or artifacts, they are tests to show that any piece of the UI can be animated.</li><li>Other projects include <a href="https://github.com/greggirwin/red-formatting/blob/master/formatting-functions.adoc" target="_blank">format</a>, <span style="color: #6aa84f; font-family: courier;"><b>split</b></span>, HOFs, and modules, each with a great deal of design work and thought put into them. As an example, look at Boris' <a href="https://github.com/greggirwin/red-hof/tree/master/code-analysis" target="_blank">HOF analysis</a>. They are large and important pieces, based on historical and contemporary research, but not something we will just drop into Red, though we could. A simple <span style="color: #6aa84f; font-family: courier;"><b>map</b></span> function is a no-brainer, and could have been there day one. But that's not how we work. It's not a contest to see how many features we can add, or how fast; but how we can move software forward, make things easier, and push the state of the art. Not just in technical features (the engineering part), but in the design of a language and its ecosystem.</li></ul><h2 style="text-align: left;">Not Everyone Has These Problems</h2><div>An important aspect of Red is being self-contained. We talk about this a lot. Yes, we're considering LLVM as a target, but that has a big cost, not just benefits. Using our own compiler for everything also has costs, like slowing the move to 64-bit which is an issue for Mac users now. Workarounds like VMs and Docker containers are just that. We want things to be easy for you, but that doesn't mean they're easy for us. Here's an example.</div><div><br /></div><div>Boris found a <a href="https://github.com/red/red/issues/4875" target="_blank">bug related to printing time values</a> in Red/System. @dockimbel finally tracked it down, and posted this investigative report:<br /></div><div><p style="background-color: white; box-sizing: border-box; color: #24292e; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"; font-size: 14px; margin-bottom: 16px; margin-top: 0px !important;"><a class="user-mention" data-hovercard-type="user" data-hovercard-url="/users/hiiamboris/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/hiiamboris" style="background-color: transparent; box-sizing: border-box; font-weight: 600; text-decoration-line: none; white-space: nowrap;"></a></p><blockquote><p style="background-color: white; box-sizing: border-box; color: #24292e; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"; font-size: 14px; margin-bottom: 16px; margin-top: 0px !important;"><a class="user-mention" data-hovercard-type="user" data-hovercard-url="/users/hiiamboris/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/hiiamboris" style="background-color: transparent; box-sizing: border-box; font-weight: 600; text-decoration-line: none; white-space: nowrap;">@hiiamboris</a> It was a (R/S) compiler issue afterall. ;-)<code style="background-color: var(--color-markdown-code-bg); border-radius: 6px; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0.4em;">size? a</code> was the guilty part. The compiler was wrongly generating code for loading <code style="background-color: var(--color-markdown-code-bg); border-radius: 6px; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0.4em;">a</code> even though <code style="background-color: var(--color-markdown-code-bg); border-radius: 6px; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0.4em;">size?</code> is statically evaluated by the compiler and replaced by a static integer value. Given that <code style="background-color: var(--color-markdown-code-bg); border-radius: 6px; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0.4em;">a</code> was a float type, its value was pushed onto the x87 FPU stack, but never popped. That stack has a 7 slots limit. Running the loop 5 times was enough to leave only 2 slots free. When the <a href="https://github.com/red/red/blob/master/runtime/dtoa.reds#L874" style="background-color: transparent; box-sizing: border-box; text-decoration-line: none;">big float expression</a> is encountered in <code style="background-color: var(--color-markdown-code-bg); border-radius: 6px; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, "SF Mono", Menlo, Consolas, "Liberation Mono", monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0.4em;">dtoa</code> library, it requires 3 free slots on the FPU stack, which fails and results in producing a NaN value, which wreaks havoc in the rest of the code.</p><p style="background-color: white; box-sizing: border-box; color: #24292e; font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji"; font-size: 14px; margin-bottom: 16px; margin-top: 0px;"></p></blockquote><div style="box-sizing: border-box; margin-bottom: 16px; margin-top: 0px;">The fix in the compiler was trivial (<span style="color: #6aa84f; font-family: courier;"><b>fetch-expression/final</b></span> vs <span style="color: #6aa84f; font-family: courier;"><b>fetch-expression</b></span>) but getting to that point was not. Understanding machine architectures at the lowest levels isn't for everyone, but even though our compiler code will be rewritten in the future, it's small and maintainable today. If we rely on GCC, Clang, or other compilers, hitting a bug may mean hitting a wall. So while there are costs to using our own compiler, there are also costs to depending on others. Robert Heinlein popularized <a href="https://en.wikipedia.org/wiki/There_ain%27t_no_such_thing_as_a_free_lunch" target="_blank">TANSTAAFL</a>, but the concept is not science fiction. As a side note, just as we moved from GDI+ to D2D, x87 for float support was an early choice meant to support older platforms and we are planning to switch to SSE.</div><div style="box-sizing: border-box; margin-bottom: 16px; margin-top: 0px;">If compilers are your thing, or you like system level programming, join our community and get to know Red/System. See how our toolchain works, and consider joining us.<br /><h2 style="text-align: left;">The Big Picture</h2></div></div><div>I just read <span style="color: #6aa84f; font-family: helvetica;">101 things I learned in architecture school</span><span style="font-family: inherit;"> (which I heard about via <a href="https://www.youtube.com/watch?v=AbgsfeGvg3E" target="_blank">Kevlin Henney</a>, (though it may not have been that specific talk) and what struck me the most about it is how we've commandeered the word "architecture" for software but <i>completely removed</i> the human aspect. An architect does so much more than we do. Software architects are really structural engineers. If a single developer builds a complete app, they <i>have to</i> do the UI. They engineer a building and then slap on whatever sheathing is at hand, cutting doors and windows without concern for their location. And the app is viewed in isolation, as if it's the only thing a user has on their system, without consideration for its site, context, or relationships. What makes real architecture hard (and why the author notes that many architects hit their stride when they are older), is that you have to know so much. So many considerations, disciplines, and constraints are involved, and you have to unify them. It's both creative and scientific. What makes great architecture great is that it makes your experience better. Maybe even wonderful. If we only think about the mechanical aspects, our software will never be <i>beautiful</i>.</span></div><div><br /></div><div>We haven't articulated this view for what we do, I think because we didn't realize it. At least I didn't. We talk about the whole being greater than the sum of its parts, and not just making everything libraries so it feels more natural and <i>less</i> mechanical. How a REPL and single exe make it easier to get started, and not having to use many tools is better. But we haven't explicitly said "Here's how it's laid out, and why. Here's how it's put together; these are the critical elements. Here's what it looks like from a distance, and when you enter its space." Implicitly we do that every day, through the work, but we don't <i>talk</i> about it. Or only once a year.</div><p></p>Unknownnoreply@blogger.com9tag:blogger.com,1999:blog-5936111837781935054.post-86522865357502683392020-08-20T12:54:00.000+02:002020-08-20T12:54:19.630+02:00Red/System: New Features<p>
In the past months, many new features were added to Red/System, the low-level
dialect embedded in Red. Here is a sum up if you missed them.
</p>
<h2 style="text-align: left;">Subroutines</h2>
<p>
During the work on the low-level parts of the
<a href="https://www.red-lang.org/2020/08/a-new-fast-and-flexible-lexer.html" target="_blank">new Red lexer</a>, the need arised for intra-function factorization abilities to keep the
lexer code as
<a href="https://en.wikipedia.org/wiki/Don%27t_repeat_yourself" target="_blank">DRY</a>
as possible. Subroutines were introduced to solve that. They act as the
<a href="https://www.c64-wiki.com/wiki/GOSUB" target="_blank">GOSUB</a>
directive from Basic language. They are defined as a separate block of code
inside a function's body and are called like regular functions (but without any
arguments). So they are much lighter and faster than real function calls and
require just one slot of stack space to store the return address.
</p>
<p>The declaration syntax is straightforward:</p>
<pre class="code"> <name>: [<body>]
<name> : subroutine's name (local variable).
<body> : subroutine's code (regular R/S code).
</pre>
<p>
To define a subroutine, you need to declare a local variable with the
subroutine! datatype, then set that variable to a block of code. You can then
invoke the subroutine by calling its name from anywhere in the function body
(but after the subroutine own definition).
</p>
<p>Here is a first example of a fictive function processing I/O events:</p>
<pre class="code"> process: func [buf [byte-ptr!] event [integer!] return: [integer!]
/local log do-error [subroutine!]
][
log: [print-line [">>" tab e "<<"]]
do-error: [print-line ["** Error:" e] return 1]
switch event [
EVT_OPEN [e: "OPEN" log unless connect buf [do-error]]
EVT_READ [e: "READ" log unless receive buf [do-error]]<br /> EVT_WRITE [e: "WRITE" log unless send buf [do-error]]<br /> EVT_CLOSE [e: "CLOSE" log unless close buf [do-error]]<br /> default [e: "<unknown>" do-error]
]
0
]
</pre>
<p>
This second example is more complete. It shows how subroutines can be combined
and how values can be returned from a subroutine:
</p>
<pre class="code"> #enum modes! [
CONV_UPPER
CONV_LOWER
CONV_INVERT
]
convert: func [mode [modes!] text [c-string!] return: [c-string!]
/local
lower? upper? alpha? do-conv [subroutine!]
delta [integer!]
s [c-string!]
c [byte!]
][
lower?: [all [#"a" <= c c <= #"z"]]
upper?: [all [#"A" <= c c <= #"Z"]]
alpha?: [any [lower? upper?]]
do-conv: [s/1: s/1 + delta]
delta: 0
s: text
while [s/1 <> null-byte][
c: s/1
if alpha? [
switch mode [
CONV_UPPER [if lower? [delta: -32 do-conv]]
CONV_LOWER [if upper? [delta: 32 do-conv]]
CONV_INVERT [delta: either upper? [32][-32] do-conv]
default [assert false]
]
]
s: s + 1
]
text
]
probe convert CONV_UPPER "Hello World!"
probe convert CONV_LOWER "There ARE 123 Dogs."
probe convert CONV_INVERT "This SHOULD be INVERTED!"
</pre>
<p>will output:</p>
<pre class="code"> HELLO WORLD!
there are 123 dogs.
tHIS should BE inverted!
</pre>
<p>
Support for getting a subroutine address and dispatching dynamically on it is
planned to be added in the future (something akin
<a href="https://en.wikipedia.org/wiki/Goto#Computed_GOTO_and_Assigned_GOTO" target="_blank">computed GOTO</a>). More examples of subroutines can be found in the new lexer code, like in
the
<span style="font-family: courier;"><a href="https://github.com/red/red/blob/master/runtime/lexer.reds#L1689" target="_blank">load-date</a></span>
function.
</p>
<h2 style="text-align: left;">New system intrinsics</h2>
<div><br /></div>
<div>
Several new extensions to the
<a href="https://static.red-lang.org/red-system-specs.html#section-13" target="_blank">system</a>
path have been added.
</div>
<div><br /></div>
<h3 style="text-align: left;">Lock-free atomic intrinsics</h3>
<div><br /></div>
<div>
A simple low-level OS threads
<a href="https://github.com/red/red/blob/master/runtime/threads.reds" target="_blank">wrapper API</a>
has been added internally to the Red runtime as preliminary work on supporting
parts of IO concurrency and parallel processing in the future. In order to
complement it, a set of atomic intrinsics were added to enable the
implementation of
<a href="https://preshing.com/20120612/an-introduction-to-lock-free-programming/" target="_blank">lock-free</a>
and wait-free
<a href="https://en.wikipedia.org/wiki/Non-blocking_algorithm" target="_blank">algorithms</a>
in a multithreaded execution context.
</div>
<div><br /></div>
<div>
The new atomic intrinsics are all documented
<a href="https://static.red-lang.org/red-system-specs.html#section-13.24" target="_blank">here</a>. Here is a quick overview:
</div>
<div>
<ul style="text-align: left;">
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/atomic/fence</b></span>: generates a read/write data memory barrier.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/atomic/load</b></span>: thread-safe atomic read from a given memory location.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/atomic/store</b></span>: thread-safe atomic write to a given memory location.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/atomic/cas</b></span>: thread-safe atomic compare&swap to a given memory location.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/atomic/<math-op></b></span>: thread-safe atomic math or bitwise operation to a given memory location
(<span style="font-family: courier;">add, sub, or, xor, and</span>).
</li>
</ul>
</div>
<div><br /></div>
<div>Other new intrinsics</div>
<div>
<ul style="text-align: left;">
<li>
<span style="color: #6aa84f; font-family: courier;"><b> system/stack/allocate/zero</b></span>: allocates a storage space on stack and zero-fill it.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/stack/push-all</b></span>: saves all registers to stack.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/stack/pop-all</b></span>: restores all registers from stack.
</li>
<li>
<span style="color: #6aa84f; font-family: courier;"><b>system/fpu/status</b></span>: retrieves the FPU exception bits status as a 32-bit integer.
</li>
</ul>
</div>
<div><br /></div>
<h2 style="text-align: left;">Improved literal arrays</h2>
<p>
The main change is the removal of the hidden size inside the
<span style="font-family: courier;">/0</span> index slot. The size of a
literal array can now only be retrieved using the
<span style="color: #6aa84f; font-family: courier;"><b>size?</b></span>
keyword, which is resolved at compile time (rather than run-time for
<span style="font-family: courier;">/0</span>
index access).
</p>
<p>
A notable addition is the support for
<a href="file:///C:/Dev/Red/docs/red-system/red-system-specs-light.html#section-4.8.7" target="_blank">binary arrays</a>. Those arrays can be used to store byte-oriented tables or embed arbitray
binary data into the source code. For example:
</p>
<pre class="code"> table: #{0042FA0100CAFE00AA}
probe size? table ;-- outputs 9
probe table/2 ;-- outputs "B"
probe as integer! table/2 ;-- outputs 66</pre>
The new Red lexer code uses them
<a href="https://github.com/red/red/blob/master/runtime/lexer.reds#L102" target="_blank">extensively</a>.
<div>
<br />
<h2 style="text-align: left;">Variables and arguments grouping</h2>
<p>
It is now possible to group the type declaration for local variables and
function arguments. For example:
</p>
<pre class="code"> foo: func [
src dst [byte-ptr!]
mode delta [integer!]
return: [integer!]
/local
p q buf [byte-ptr!]
s1 s2 s3 [c-string!]
]
</pre>
<p>
Note that the compiler supports those features through code expansion at
compile time, so that error reports could show each argument or variable
having its own type declaration.
</p>
<h2 style="text-align: left;">Integer division handling</h2>
<p>
Integer division handling at low-level has notorious shortcomings with
different handling for each edge case depending on the hardware platform.
Intel IA-32 architecture tends to handle those cases in a slightly
safer way, while ARM architecture produces erroneous results silently typically for the following two cases:
</p>
<p></p>
<ul style="text-align: left;">
<li>division by zero</li>
<li>
division overflow (<span style="font-family: verdana;">-2147483648 / -1</span>)
</li>
</ul>
<p></p>
<p>
IA-32 CPU will generate an exception, while ARM ones will return invalid
results (respectively 0 and -2147483648). This makes it difficult to produce
code that will behave the same on both architectures when integer divisions
are used. In order to reduce this gap, R/S compiler will now generate extra
code to detect those cases for ARM targets and raise a runtime exception.
Such extra checkings for ARM are produced only in debug compilation mode. In
release mode, priority is given to performance, no runtime exception will
occur in such cases on ARM (as the overhead is significant). So, be sure to check your code on ARM platform
thoroughly in debug mode before releasing it. This is not a perfect
solution, but at least, it makes it possible to detect those cases through
testing in debug mode.
</p>
<h2 style="text-align: left;">Others</h2>
<p>Here is a list of other changes and fixes in no particular order:</p>
<ul style="text-align: left;">
<li>
<p>
Cross-referenced aliased fields in structs defined in same context are
now allowed. Example:
</p>
<pre class="code"> a!: alias struct! [next [b!] prev [b!]]
b!: alias struct! [a [a!] c [integer!]]<br /></pre>
</li>
<li>
<span style="font-family: courier;">-0.0</span> special float literal
is now supported.
</li>
<li>
<span style="font-family: courier;">+1.#INF</span><span> </span>is
also now supported as valid literal in addition to<span> </span><span style="font-family: courier;">1.#INF</span><span> </span>for
positive infinite.
</li>
<li>Context-aware get-words resolution.</li>
<li>New <span style="color: #6aa84f; font-family: courier;"><b>#inline</b></span> <a href="https://static.red-lang.org/red-system-specs.html#section-16.11" target="_blank">directive</a> to inline assembled binary code.</li>
<li>Dropped support for <span style="font-family: courier;">%</span> and <span style="font-family: courier;">//</span> operators on float types, as they were relying
on FPU's relative support, the results were not reliable across platforms.
Use <span style="color: #6aa84f; font-family: courier;"><b>fmod</b></span> function instead from now on.
</li>
<li>Added <span style="color: #6aa84f; font-family: courier;"><b>--show-func-map</b></span> compilation option: when used, it will output a map
of R/S function addresses/names, to ease low-level debugging.
</li>
<li>FIX: issue #4102: ASSERT false doesn't work.</li>
<li>
FIX: issue #4038: cast integer to float32 in math expression gives
wrong result.</li>
<li>
FIX: byte! to integer! conversion not happening in some cases.
Example: <span style="font-family: courier;">i: as-integer (p/1 - #"0")
</span></li>
<li>
FIX: compiler state not fully cleaned up after premature termination. This
affects multiple compilation jobs done in the same Rebol2 session,
resulting in weird compilation errors.</li>
<li>
FIX: issue #4414: round-trip pointer casting returns an incorrect
result in some cases.</li>
<li>
FIX: literal arrays containing true/false words could corrupt the
array. Example: <span style="font-family: courier;">a: ["hello" true "world" false]
</span></li>
<li>FIX: improved error report on bad <span style="color: #6aa84f; font-family: courier;"><b>declare </b></span>argument.</li>
</ul>
<p></p>
<p></p>
<div></div>
<p></p>
</div>
Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com7tag:blogger.com,1999:blog-5936111837781935054.post-12004914061138269492020-08-03T14:06:00.002+02:002020-08-03T16:05:19.670+02:00A New Fast and Flexible Lexer<div>
A programming language lexer is the part in charge of converting textual code
representation into a structured memory representation. In Red, it is
accomplished by the
<font color="#41b375" face="courier"><b>load </b></font>function, which calls
the lower-level <font color="#41b375" face="courier"><b>transcode</b></font>
native. Until now, Red was relying on a lexer entirely written using the
<a href="https://www.red-lang.org/2013/11/041-introducing-parse.html" target="_blank">Parse dialect</a>. Though, the parsing rules were constructed to be easily maintained and not
for performance. Rewriting those rules to speed them up could have been
possible, but rewriting the lexer entirely in Red/System would give the
ultimate performance. It might not matter for most user scripts, but given
that Red is also a data format, we need a solution for fast (near-instant)
loading of huge quantities of Red values stored in files or transferred
through the network.
</div>
<div><br /></div>
<div>The new lexer main features are:</div>
<div>
<ul style="text-align: left;">
<li>
<b>High performance</b>, typically <u>50 to 200 times</u> faster than the
older one.
</li>
<li>
New<b> scanning features</b>: identify values and their datatypes
without loading them.
</li>
<li>
<b>Instrumentation</b>: customize the lexer's behavior at will using
an event-oriented API.
</li>
</ul>
</div>
<p style="text-align: left;">
The reference documentation is available
<a href="https://github.com/red/docs/blob/master/en/lexer.adoc" target="_blank">there</a>. This new lexer is available in Red's auto-builds since June.
</p>
<h2 style="text-align: left;">Performance</h2>
<div><br /></div>
<div>
Vastly increased performance is the main driver for this new lexer. Here is a
little benchmark to let you appreciate how far it gets.
</div>
<div><br /></div>
<div>The benchmarking tasks are:</div>
<div>
<ul style="text-align: left;">
<li>
100 x compiler.r: loads 100 times compiler.r
<a href="https://github.com/red/red/blob/master/compiler.r" target="_blank">source file</a>
from memory (~126KB, so about ~12MB in total).
</li>
<li>
1M short integers: loads a string of 1 million
<font face="courier">`1`</font> separated by a space.
</li>
<li>
1M long integers: loads a string of 1 million
<font face="courier">`123456789`</font> separated by a space.
</li>
<li>
1M dates: loads a string of 1 million
<font face="courier">`26/12/2019/10:18:25`</font> separated by a
space.
</li>
<li>
1M characters: loads a string of 1 million
<font face="courier">`#"A"`</font> separated by a space.
</li>
<li>
1M escaped characters: loads a string of 1 million
<font face="courier">`#"^(1234)"`</font> separated by a space.
</li>
<li>
1M words: loads a string of 1 million
<font face="courier">`random "abcdefghijk"`</font> separated by a
space.
</li>
<li>
100K words: loads a string of 100 thousands `<font face="courier">random "abcdefghijk"`</font>
separated by a space.
</li>
</ul>
</div>
<div><br /></div>
<div>And the results are (on a Core i7-4790K):</div>
<pre class="code"> Loading Task v0.6.4 (sec) Current (sec) Gain factor
---------------------------------------------------------------------
100 x compiler.r 41.871 0.463 90
1M short integers 14.295 0.071 201
1M long integers 18.105 0.159 114
1M dates 29.319 0.389 75
1M characters 14.865 0.092 162
1M escaped characters 14.909 0.120 124
1M words n/a 1.216 n/a
100K words 23.183 0.070 331
</pre>
<div><u>Notes</u>: </div>
<div><br /></div>
<div>
- Only <font color="#41b375" face="courier"><b>transcode</b></font> is used in the loading tasks (<font face="courier">system/lexer/transcode</font> in 0.6.4).
</div>
<div><br /></div>
<div>
- The "1M words" task fails on 0.6.4 as the symbol table expansion time is
exponential due to some hashtable bugs. That also explains the big gap for the
"100K words" task. Those issues are fixed in the current version and the
symbol table further optimized for speed. Though, the execution time increase
between 100K and 1M words tests in new lexer is not linear which may be
explained by a high number of collisions in the internal hashtable due to
limited input variability.
</div>
<div><br /></div>
<div>
- The 0.6.4's lexer can only process strings as input, while the new lexer
only processes internally only UTF-8 binary inputs. The input strings were
converted to the lexer's native format in order to more accurately compare
their speed. Providing a string instead of a binary series as input to the new
lexer incurs on average a ~10% speed penalty.
</div>
<div><br /></div>
<h2 style="text-align: left;">Scanning</h2>
<div><br /></div>
<div>
It is now possible to only scan tokens instead of loading them. Basically,
that means identifying a token's length and type without loading it (so
without requiring extra memory and processing time). This is achieved by using
the new <b><font color="#41b375" face="courier">scan </font></b>native.
</div>
<pre class="code"> >> scan "123"
== integer!
>> scan "w:"
== set-word!
>> scan "user@domain.com"
== email!
>> scan "123a"
== error!<br /></pre>
<br />
<div>
It is possible to achieve even higher scanning speed by giving up a bit on
accuracy. That is the purpose of the <b><font color="#41b375" face="courier">scan/fast</font></b>
refinement. It trades maximum performance for type recognition accuracy. You
can find the list of "guessed" types in the table
<a href="https://github.com/red/docs/blob/master/en/lexer.adoc#scan" target="_blank">there</a>.
</div>
<div><br /></div>
<pre class="code"> >> scan/fast "123"
== integer!
>> scan/fast "a:"
== word!
>> scan/fast "a/b"
== path!
</pre>
<div><br /></div>
<div>
Scanning applies to the first token in the input series. When an iterative
application is needed in order to scan all tokens from a given input, the
<b><font color="#41b375" face="courier">/next</font></b> refinement can be
used for that. It will return the input series past the current token allowing
to get the precise token size in the input string. It can be used in
combination with <b><font color="#41b375" face="courier">/fast</font></b> if
required. For example:
</div>
<div>
<pre class="code"> src: "hello 123 world 456.789"
until [
probe first src: scan/next src
empty? src: src/2
]</pre>
<div>Outputs:</div>
<pre class="code"> word!
integer!
word!
float!</pre>
</div>
<div><br /></div>
<h2>Matching by datatype in Parse</h2>
<div><br /></div>
<div>
The new lexer enables also matching by datatype directly from Parse dialect.
Though, this feature is limited to binary input only.
</div>
<pre class="code"> >> parse to-binary "Hello 2020 World!" [word! integer! word!]
== true
>> parse to-binary "My IP is 192.168.0.1" [3 word! copy ip tuple!]
== true
>> ip
== #{203139322E3136382E302E31}
>> load ip
== 192.168.0.1
</pre>
<div>
Notice that the whitespaces in front of tokens are skipped automatically in this
matching mode.</div>
<div><br /></div>
<h2 style="text-align: left;">Instrumentation</h2>
<div><br /></div>
<div>
Lexers in Red and Rebol world used to be black boxes, this is no longer the
case with Red's new lexer and its tracing capabilities. It is now possible to
provide a callback function that will be called upon lexer events triggered
while parsing tokens. It gives deeper control to users, for example allowing
to:
</div>
<div>
<ul style="text-align: left;">
<li>
Trace the behavior of the lexer for debugging or statistical purposes.
</li>
<li>Catch errors and resume loading by skipping invalid data.</li>
<li>
On-the-fly input transformation (to remove/alter some non-loadable parts).
</li>
<li>Extend the lexer with new lexical forms.</li>
<li>Process serialized Red data without having to fully load the input.</li>
<li>Extract line comments that would be lost otherwise.</li>
</ul>
</div>
<div><br /></div>
<div>
Lexer's tracing mode is activated by using the
<font color="#41b375"><font face="courier"><b>/</b></font><b style="font-family: courier;">trace</b> </font>refinement on <font color="#41b375" face="courier"><b>transcode</b></font>. The syntax is:
</div>
<pre class="code"> transcode/trace <input> <callback>
<input> : series to load (binary! string!).
<callback> : a callback function to process lexer events (function!).
</pre>
<div>
That function is called on specific events generated by the lexer:
<font face="courier">prescan, scan, load, open, close, error</font>. The
callback function and events specification can be found <a href="https://github.com/red/docs/blob/master/en/lexer.adoc#transcodetrace" target="_blank">there</a>.
</div>
<div><br /></div>
<div>
A default tracing callback is provided in
<font color="#41b375" face="courier"><b>system/lexer/tracer</b></font>:
</div>
<pre class="code"> >> transcode/trace "hello 123" :system/lexer/tracer
prescan word 1x6 1 " 123"
scan word 1x6 1 " 123"
load word hello 1 " 123"
prescan integer 7x10 1 ""
scan integer 7x10 1 ""
load integer 123 1 ""
== [hello 123]
</pre>
<div>
That tracing function will simply print the lexer event info. If a syntax
error occurs, it will cancel it and resume on the next character after the
error position.
</div>
<div><br /></div>
<div>
Several more sophisticated examples can be found on our
<a href="https://github.com/red/code/tree/master/Scripts/lexer" target="_blank">red/code repository</a>.
</div>
<div><br /></div>
<h2 style="text-align: left;"><br /></h2>
<h2 style="text-align: left;">Implementation notes</h2>
<div><br /></div>
<div>
This new lexer has been specifically prototyped and designed for performance.
It relies on a token-oriented pipelined approach consisting of 3 stages:
prescanning, scanning and loading.
</div>
<div><br /></div>
<div>
Prescanning is achieved using only a
<a href="https://github.com/red/red/blob/master/runtime/lexer.reds#L2114" target="_blank">tight loop</a>
and a state machine (<a href="https://en.wikipedia.org/wiki/Finite-state_machine" target="_blank">FSM</a>). The loop reads UTF-8 encoded input characters one byte at a time. Each
byte is identified as part of a <a href="https://github.com/red/red/blob/master/runtime/lexer.reds#L172" target="_blank">lexical class</a>. The lexical class is then used to transition from one state to another in
the
<a href="https://github.com/red/red/blob/master/docs/lexer/lexer-states.txt" target="_blank">FSM</a>, using a big
<a href="https://github.com/red/red/blob/master/runtime/lexer-transitions.reds#L112" target="_blank">transition table</a>. Once a terminal state (<a href="https://github.com/red/red/blob/master/runtime/lexer-transitions.reds#L4" target="_blank">state names</a>
with a `T_` prefix) or input's end is reached, the loop exits, leading to the
next stage. The result of the prescanning stage is to locate a token begin/end
positions and give a pretty accurate guess about the token's datatype. It can
also detect some syntax errors if the FSM cannot reach a proper datatype
terminal state. This approach provides the fastest possible speed for tokens
detection, but it cannot be fully accurate, nor can it validate deeply the
token content for some complex types (e.g. dates).
</div>
<div><br /></div>
<div>
Adding more states would provide greater accuracy and cover more syntatic
forms, but at the cost of growing the transition table a lot due to the need to duplicate many state. Currently the table weights 2440 bytes, which is already quite
big to be kept entirely in the CPU data cache (usually 8, 16 or 32KB per core, the
lexical table uses 1024 bytes and there two other minor tables used in the
tight loop). The data cache also needs to handle the parsed input data and
part of the native stack, so the available space is limited.
</div>
<div><br /></div>
<div>
The tight loop code is also optimized for keeping <a href="https://en.wikipedia.org/wiki/Branch_predictor" target="_blank">branch mispredictions</a> as low as possible. It currently
only relies on two branchings. The loop code could be also further reduced by, for
example, pre-multiplying the state values to avoid the multiplication when
calculating the table entry offset. Though, we need to wait for a fully
optimizing code generation backend before trying to extract more performance
from that loop code, or we might be taking wrong directions.
</div>
<div><br /></div>
<div>
Scanning stage happens when a token has been identified. It consists in
eventually calling a scanner function to deep-check the token for errors and
more accurately determine the datatype. Loading stage then follows (unless
only scanning was requested by the user). It will eventually call a loader
function that will construct the Red value out of the token. In case of
any-block series, the scanners will actually do the series construction on
reaching the ending delimiter (which requires
<a href="https://github.com/red/red/blob/master/runtime/lexer.reds#L2189" target="_blank">special handling</a>
for paths), so no loader is needed there. Conversely, loaders can be invoked
in validating mode only (not constructing the value), in order to avoid code
duplication when complex code is required for decoding/validating the
token (e.g. date!, time!, strings with UTF-8 decoding,...).
</div>
<div><br /></div>
<div>
For the record, there was an
<a href="https://github.com/red/red/tree/fast-lexer-date" target="_blank">attempt</a>
at creating specific FSM for date! and time! literal forms parsing, to reduce
the amount of rules that need to be handled by pure code. The results were not
conclusive, as the amount of code required for special case handling was
still significant and the performance of the FSM parsing loop was below the
current pure code version. This approach can be reexamined once we get the
fully optimizing backend.
</div>
<div><br /></div>
<div>
The FSM states, lexical classes and transitions are documented in
<a href="https://github.com/red/red/blob/master/docs/lexer/lexer-states.txt" target="_blank">lexer-states.txt</a>
file. A simple syntax is used to describe the transitions and possible
branching from one state to others. The FSM has three possible entry points:
<font face="courier">S_START</font>, <font face="courier">S_PATH</font> and
<font face="courier">S_M_STRING</font>. Parsing path items requires specific
states even for common types. For curly-braced strings, it is necessary to
exit the FSM on each occurrence of open/close curly braces in order to count
the nested ones and accurately determine where it ends. In both those path and
string cases, the FSM needs to be re-entered in a different state than
S_START.
</div>
<div><br /></div>
<div>
In order to build the FSM transition table, there is a workflow that goes from that lexer-states.txt file to the final transition
table data in binary. It basically looks like this:
</div>
<pre class="code"> FSM graph -> Excel file -> CSV file -> binary table</pre>
<div>The more detailed steps are:</div>
<div>
<ol style="text-align: left;">
<li>
Manually edit changes in the <a href="https://github.com/red/red/blob/master/docs/lexer/lexer-states.txt" target="_blank">lexer-states.txt</a> file.
</li>
<li>
Port those changes into the
<a href="https://github.com/red/red/blob/master/docs/lexer/lexer-FSM.xlsx" target="_blank">lexer.xlsx</a>
file by properly setting the transition values.
</li>
<li>
Save that Excel table in CSV format as
<a href="https://github.com/red/red/blob/master/docs/lexer/lexer-FSM.csv" target="_blank">lexer.csv</a>.
</li>
<li>
Run the
<a href="https://github.com/red/red/blob/master/utils/generate-lexer-table.red" target="_blank">generate-lexer-table.red</a>
script from Red repo root folder. The
<a href="https://github.com/red/red/blob/master/runtime/lexer-transitions.reds" target="_blank">lexer-transitions.reds</a>
file is regenerated.</li>
</ol>
</div>
<div>
The lexer code relies on several other tables for specific types handling like
path ending detection, floating point numbers syntax validation, binary series
and escaped characters decoding. Those tables are either manually written (not
planned to be ever changed) or generated using
<a href="https://github.com/red/red/blob/fast-lexer-date/utils/generate-misc-tables.red" target="_blank">this script</a>.
</div>
<div><br /></div>
<div>Various other points worth mentioning:</div>
<div><br /></div>
<div>
- The lexer works natively with UTF-8 encoded binary buffers as input. If a
string! is provided as input, there is an overhead for converting internally
such string to binary before passing it to the lexer. A unique internal buffer
is used for those conversions with support for recursive calls.
</div>
<div><br /></div>
<div>
- The lexer uses a single accumulative cells buffer for storing loaded values,
with an inlined any-block stack.
</div>
<div><br /></div>
<div>
- The lexer and lexer callbacks are fully recursive and GC-compliant.
Currently callbacks can be function! only, this can be extended in the future
to support routines also for much faster processing.
</div>
<div><br /></div>
Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com2tag:blogger.com,1999:blog-5936111837781935054.post-18522564273591299982020-03-20T20:00:00.001+01:002020-03-20T20:02:25.639+01:00GTK, fast lexer, money, deep testing, and our first commercial productIt's been a busy start to the year for the team. Work has continued on many fronts, and we have some new team members helping us to keep the momentum going. We announced in January that GTK and the Fast Lexer were close, and they are even closer now. The hard part about making announcements is that some of this work is unpredictable and changes in scope after we do. Or the world steps in and a pandemic throws a wrench into your plans.<br />
<br />
@bitbegin has done an enormous amount of work on the GTK branch. You can check out the <a href="https://github.com/red/red/commits/GTK">GTK branch</a> to see some of what goes into supporting a new GUI system. The more features we include in Red, the more that have to be ported and maintained. Unfortunately, most operating systems and UI systems have large, complicated sets of APIs and interactions.<br />
<br />
Because GUI systems are so complex, and Red not only has to handle them, but also adds its own reactive framework, there are more places for bugs to hide. And users are involved, which is the worst part. They do nothing but cause problems. To that end, in addition to his normal deep diving and bug hunting, @hiiamboris has been working on an automated <span style="color: lime; font-family: "courier new" , "courier" , monospace;">view</span> test system, which is no small feat. @9214 has joined him on the hunt, and we will squash a good number of bugs for our next release.<br />
<br />
The fast lexer was in near-final testing when we decided that it was worth delaying its merge in order to incorporate some new lexical forms that we had planned to include. Then we looked at some old tickets related to modulo and division operators, and a couple lexing questions came up related to <span style="color: lime; font-family: "courier new" , "courier" , monospace;">tag!</span>. Suddenly the fast lexer work was back in code mode. New lexical forms usually means new datatypes, and that's the case here.<br />
<br />
<h3>
New Datatypes</h3>
One we've expected for some time, and thanks to @9214 it's now a reality. <span style="color: lime; font-family: "courier new" , "courier" , monospace;">Money!</span> is coming. There is a <a href="https://github.com/red/red/commits/money">branch</a> for it, but no need to comment at this time. There are a few features, like <span style="color: lime; font-family: "courier new" , "courier" , monospace;">round</span> still to be completed, but the bulk of the work is done. @BeardPower did some great experimental work that we thought might be used for money, based on Douglas Crockford's Dec64 design, but until Red is fully 64-bit it was only Dec32, and the limited range was determined not to be enough. That work won't go to waste though, it's just waiting for its moment in the sun. The current version of <span style="color: lime; font-family: "courier new" , "courier" , monospace;">money! </span><span style="font-family: inherit;">is a BCD implementation, but that shouldn't concern anyone outside the core team. What you care about is that it can be used for accurate financial calculations that don't suffer from floating point errors. It will also support an optional currency identifier, e.g. USD or EUR, and automatic group separators when </span><span style="color: lime; font-family: "courier new" , "courier" , monospace;">form</span><span style="font-family: inherit;">ed.</span><br />
<br />
Another new type is called <span style="color: lime; font-family: "courier new" , "courier" , monospace;">ref!</span> and is still being designed. The basic concept is simple: <span style="color: lime; font-family: "courier new" , "courier" , monospace;">@reference </span>is a form most people are familiar with today, just as hashtags are known (though we use the historical name<span style="color: lime; font-family: "courier new" , "courier" , monospace;"> issue!</span>, because the new name hadn't become part of our global lexicon back then). Issues, in Rebol 2, were a string type, but Red made them a word type instead. That has benefits (mainly efficiency), but also costs (symbol table space and lexical limitations). For instance, in R2 you could say<span style="color: lime; font-family: "courier new" , "courier" , monospace;"> #abc:123</span>, but not so in Red. Life is compromise. <span style="color: lime; font-family: "courier new" , "courier" , monospace;">Ref!</span> will be a string type, making it quite flexible. While we most often think of them as referring to a person, they can refer to, for example, a location in a file. You can do that with strings too, of course, but that's the beauty of rich datatypes. By using a <span style="color: lime; font-family: "courier new" , "courier" , monospace;">ref!</span>, you can build rich dialects and make your intent clear in the data itself.<br />
<br />
Finally, Red is going to add a new <span style="color: lime; font-family: "courier new" , "courier" , monospace;">raw-string!</span> form. It's a combination of the raw string literals some languages support, and <a href="https://en.wikipedia.org/wiki/Here_document">heredoc</a>. The goal is to make it easier to include content that would otherwise require escape sequences that sometimes clutter inline data and lead to errors. A lot of time and effort went into the (sometimes heated) discussion around the need, use cases, and syntax. Right now this is just a new literal form, rather than a whole new datatype. They will still be strings when loaded, until we see how they are used by others and if they deserve to be a separate type.<br />
<br />
We don't add new datatypes lightly, and design choices have to keep the big picture in mind as well. Balancing the value of new types against their added complexity in the language is hard work, but satisfying if it makes everyone's life better. To that end, when these new types and features become available, we need your input on how they work, where you use them, and what gaps need to be filled.<br />
<br />
Both the fast lexer and money datatype leverage some new features in Red/System, which we'll talk about in a future post.<br />
<br />
<h3>
All Aboard!</h3>
We're also very excited to announce the imminent release of our first commercial product. We alluded to it at the beginning of the year, and it's almost ready to leave the station. There will be plenty of train-related puns, because it's a Railroad syntax diagram generator. Here's what it looks like:<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikvaPq87xlbf5jV_lNsSbOgWPk6iuAR3ZsxFUIC-tgclrFPvim0fsu2i_-P89LDJEGKBdLMjZuyOlzSLlQBYGc3NHQFGcREHl9RV2EG5ffx7b6VR5qzesUxcgg6L5fGq-pDsVKWhRnMYU/s1600/diagrammar-screenshot-json.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="787" data-original-width="1162" height="432" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEikvaPq87xlbf5jV_lNsSbOgWPk6iuAR3ZsxFUIC-tgclrFPvim0fsu2i_-P89LDJEGKBdLMjZuyOlzSLlQBYGc3NHQFGcREHl9RV2EG5ffx7b6VR5qzesUxcgg6L5fGq-pDsVKWhRnMYU/s640/diagrammar-screenshot-json.png" width="640" /></a></div>
<span id="goog_633644253"></span><span id="goog_633644254"></span>More details will come soon, but here's a short list of some of it's features:<br />
<br />
<ul>
<li>Live coded diagramming. You write your grammar and the diagram appears in real time.</li>
<li>Red <span style="color: lime; font-family: "courier new" , "courier" , monospace;">parse</span> support (of course), but also support for foreign grammars like ABNF, McKeeman, and YACC. That's right, you can write using those grammars and get the same diagram. </li>
<li>When writing <span style="color: lime; font-family: "courier new" , "courier" , monospace;">parse</span> rules, you can use block level parsing, rather than character level.</li>
<li>Test inputs. Not only can you test a single input, but also entire directories of test files.</li>
<li>Test specific rules. Not only can you test from your top-level rule, but any rule in your grammar. You can also find where a particular rule matches part of your input.</li>
<li>Custom diagram styles, including options for a cleaner, more abstract view, different charset rendering options, and whether actions (parens) are displayed.</li>
<li>The ability to generate inputs that your grammar will recognize. This opens up many other use cases, including educational ones. Here's one of my favorites, that @toomasv created:</li>
</ul>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCF_Z8MSOvFJDNmeQfSv-dAvizj3rzQU1Qd4pxPEs1CpXErvMFMFXIfUPE01dc-rwF_f8xmyAI1wFvIwFkhmkI4svRMs9dPw_T6JCPl6gJAfbkw-yhTP0uuLWe7M7Fk2YFsmLeUVqPjP0/s1600/diagrammar-screenshot-qbf.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="787" data-original-width="1373" height="366" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhCF_Z8MSOvFJDNmeQfSv-dAvizj3rzQU1Qd4pxPEs1CpXErvMFMFXIfUPE01dc-rwF_f8xmyAI1wFvIwFkhmkI4svRMs9dPw_T6JCPl6gJAfbkw-yhTP0uuLWe7M7Fk2YFsmLeUVqPjP0/s640/diagrammar-screenshot-qbf.png" width="640" /></a></div>
Look at the sentences in the input area. Those were created by clicking the <i>generate</i> button. It's like parsing in reverse. So if you're not sure what inputs your grammar might recognize, or want to see examples, this lets you view things in a whole new way.<br />
<br />
<h2>
Stay Tuned</h2>
<div>
It's an exciting time to be a Reducer, and we're rolling with changes just like everyone else right now. The best way to stay healthy during this pandemic is to stay away from other people and take the Red pill.</div>
<div>
<br /></div>
<div>
Until next time.</div>
Unknownnoreply@blogger.com5tag:blogger.com,1999:blog-5936111837781935054.post-25025926247406827292020-01-01T08:23:00.000+01:002020-01-05T18:34:12.609+01:00Happy New Year!Hello and happy new year, friends of Red! We have some exciting projects we’ve been working on that will be available this year, including a new product. Let’s talk a little about what the team has been working on behind the scenes.<br />
<br />
(TL;DR: A cool <b>new product</b> with Red in 2020...plus, a robust preliminary draft of Parse documentation can now be previewed...<b>CLI library</b>...<b>fast-lexer to merge</b> soon...<b>GTK</b> on the horizon...and a new <b>native OS calendar</b> widget!)<br />
<br />
<br />
<a name='more'></a><br />
<br />
<br />
<b>Documentation for Parse: Red’s Language Construction Tool</b><br />
Our esteemed forerunner, Rebol, broke new ground with its Parse dialect, which Red has expanded on. Today, in Red, Parse has become an even more powerful built-in dialect (embedded domain-specific language) that processes input series with grammar rules in a clean and simple manner. Other language building tools exist, of course, such as Lexx and Yacc. ANTLR is a modern framework built in Java, and libraries exist for other languages, but the ease of use and power that Red’s Parse offers is unique.<br />
<br />
This isn’t a new feature, by any means, with the first public introduction <a href="https://www.red-lang.org/2013/11/041-introducing-parse.html">here</a>. Parse is easy enough to use that those basic docs have been enough, for the most part. But over time, with new users joining the <a href="https://gitter.im/red/parse">Parse chat room</a> to discuss the dialect, and it was time for full reference documentation.<br />
<br />
For the uninitiated, Parse can be used to for searching, to surface various patterns; validation, in order to confirm an input’s compliance to a specification; extraction, to sift through data and aggregate values; and modification--that is, changing the input stream itself (insertion of values, removing or transforming matched input). And Parse’s true power lies in language processing (compilers, interpreters, and lexical analyzers), particularly for DSLs; and encoding/decoding, to “translate” data formats from one to another.<br />
<br />
What makes Red’s Parse dialect a killer feature is that you can do more than parse at the character level in strings of text. That's what every other tool out there does, at best. But with Parse, you can process blocks of Red values, based on their datatype or specific literal matches, letting Red do all the work applying these layered or nested levels of analysis. By parsing blocks of values rather than character strings, we take language construction to the meta level.<br />
<br />
We’re happy to announce that <a href="https://github.com/9214">@9214</a> has completed a robust preliminary draft of Parse documentation, which you can see <a href="https://github.com/red/docs/pull/204">here</a>. Internal reviews by the core team are in process and public comment will be made possible after more analysis. In the meantime, you can view Parse itself <a href="https://github.com/red/red/blob/master/runtime/parse.reds">here</a>, and look at open issues there if you’re interested in contributing.<br />
<br />
(As a final note, the first question people often ask is how Parse compares to regular expressions. Both are DSLs, to be sure, but regex is designed for flat pattern matching of text inputs. Formats with nesting, things like matching parentheses, etc. are difficult or impossible. Engines differ widely, as well (NFA vs DFA), so your code may have very different performance in different environments. Not all regex syntaxes are compatible either. But the real difference comes in readability. Parse is more verbose, but is very clear, readable, and composable. Regex patterns are not. For fun, look up the regex to match email addresses, written by Jeffrey Friedl, in his book Mastering Regular Expressions. It’s a 6.5Kib regex. Doesn’t seem to be online anymore, but is referenced in <a href="https://stackoverflow.com/questions/201323/how-to-validate-an-email-address-using-a-regular-expression/">this chat</a>. For more discussion on this specific area, <a href="https://emailregex.com/">https://emailregex.com/</a> points to <a href="https://www.regular-expressions.info/email.html">https://www.regular-expressions.info/email.html</a> about why there’s no perfect regex for email addresses. The point is not to criticize regular expressions. They are useful, especially in the more limited forms of wildcard matching and file globbing. But they can’t recognize all grammars, and quickly become impossible for a human to read and reason about.)<br />
<br />
<b>Fast-Lexer To Be Merged Soon</b><br />
Over the last few months work on the fast-lexer branch of Red has accelerated, beginning with <a href="https://github.com/dockimbel">@dockimbel</a> prototyping the next lexer in R/S, faster than Red’s predecessors, with much smaller and simpler code, and more speed and features to come.
Issues with the current lexer had brought the need for a revision: this included the goal of making the lexer instrumentable. The requirements of a callback function with the current non-re-entrant lexer could have caused serious problems.<br />
<br />
Another hangup? An analysis of all Red's repos showed that the current lexer code was duplicated twice, beyond the basic lexing needed by load: once in the console code, once again in the VSCode plugin. The redundancy had to be eliminated.
A merge of the updated lexer into master should be expected early in 2020.<br />
<br />
<b>New Year, New Projects! </b><br />
In 2020, Red’s commercial arm will release its first product! It will be helpful to CS programs and their students, researchers, and anyone interested in analyzing the grammar and syntax of languages—which is all of you, since you're here. But it is meant to have much broader appeal as well.<br />
<br />
Every developer has probably see json.org, which brought railroad syntax diagrams to everyone’s attention. That’s right, syntax is for data formats too. But why would Douglas Crockford go to all the trouble of creating those diagrams? Communication.
Visualizations create a different experience for readers, facilitating new understanding and discussion options. This applies to those designing rich data structures (think GraphQL) and APIs, as well as languages. If you want to look professional, foster adoption, show that your design is formally specified, and help people understand it, this tool is for you.<br />
<br />
It does a lot more than create fantastic diagrams, too, to help spec and test your grammar.
It’s in invite-only beta test now, but we’ll need a limited number of testers soon, so contact us at pr@red-lang.org if you’re interested.<br />
<br />
Some of you may ask: Why is it a commercial product? Because we’re a multifaceted organization, and Open Source doesn’t pay. ;^) The Red Foundation supports the open source work done on the Red language, as well as sponsoring other work that promotes Red’s use and adoption. They’ve been mainly focused on supporting Team Red and core language work, but will expand their reach as resources allow. The first step in that direction was to get behind proving Red’s commercial viability. Redlake Technologies is a new business entity that will use Red to build tools the world needs, and will pay for, with the goal of creating a self-sustaining ecosystem of applications, services, and tokenomics that benefit both the contributors to the language and the world at large, working toward a better model of sustainable open source development.<br />
<br />
<b>By Your Command (Line)</b><br />
Also in the works: prolific community member <a href="http://hiiamboris/">@hiiamboris</a> has been burning the midnight oil creating a fantastic CLI library, which will enable authors to accept and interpret arguments and define their types easily, in an expectable, standardized manner. Command lines are the API through which a program exposes its facilities to the outside world--to users and utilities.
If you’d like to check it out, look at <a href="https://gitlab.com/hiiamboris/red-cli/">the repo</a> and see how it works! By leveraging Red’s own function spec DSL we can, with this CLI library, turn any Red function into a command line utility with one or two words.<br />
<br />
Suppose you have a `grep` function written in Red, that is to be used from the command line. Prefix it with `process-into` (the working CLI function name), add `print` for the output, compile it, and you have a standalone `grep` utility! Your feedback is important, because this design for a CLI library is quite unique.<br />
<br />
As with Red itself, you may need to let go of old ideas about how CLI libraries like `get-opt` work (though you could recreate its model easily with what Boris has built), and think like a Reducer. Just write a function with parameters, refinements, types, and doc strings, and it does the rest.<br />
<br />
<b>GTK on the Horizon </b><br />
The GTK back end is nearing completion, with just GUI console support as the last big piece to be done. <a href="https://github.com/bitbegin">@bitbegin</a> has done an enormous amount of work, with help, consultation, and testing from <a href="https://github.com/loziniak">@loziniak</a> and @<a href="https://github.com/rcqls">rcqls</a>. Once the GUI console is in place, GTK should have feature parity with MacOS.<br />
<br />
<b>Just In Time… </b><br />
The New Year will see one more surprise, in the form of a <a href="https://github.com/9214/red/tree/calendar">calendar widget</a> with support for Windows and Mac. It’ll just show up one day--a native OS widget joining you in the latest automatic build--so keep your eyes peeled.
Unknownnoreply@blogger.com14tag:blogger.com,1999:blog-5936111837781935054.post-66210110255863201832019-12-04T23:07:00.002+01:002019-12-06T20:34:59.724+01:00November 2019 in Review<i>Welcome to December, friends of Red. It's an intent and focused time of year as we wind down 2019, and the core team is making important moves (sometimes literally!) to set us up for an ambitious 2020. But first, here are just a few things that happened in November.</i><br />
<ul>
<li>First and foremost, it's always great when community members help compile resources for use by others, and we'd like to acknowledge @rebolek for his excellent compendium of historic automatic builds: <a href="https://rebolek.com/builds/">https://rebolek.com/builds/</a> (they weren't available for a hot minute, but now they're back). They can be useful if you're in need of a previous version for a specific project. Of course, you can always go <a href="https://www.red-lang.org/p/download.html">here</a> for Red's daily automated builds. But seeing as how we've a goal of being a self-sustaining, self-selecting group of do-ers, this spirit of providing collective resources is perfectly aligned with the Red-Lang we always want to be. </li>
</ul>
<ul>
<li>From the community, to outreach: thanks to community member @loziniak, <a href="https://stackshare.io/red">Red is now on Stackshare</a>, so be sure to follow us there and chime in. As a repo with 4.1k GitHub stars (and infinite possibilities), Red has a lot to offer the wider community of developers and engineers, and Stackshare is a great place to help compare and contrast us with other languages. </li>
</ul>
<ul>
<li>Now, a challenge! In the coming new year we'll be needing beta testers willing to lend their expertise in refining a new product built with Red. You read that right! If you think you'd like to be one of the contributors to spearhead a move into our next phase, we want YOU! Drop a line to @greggirwin to get in on the ground floor.<br /></li>
<li>An appreciation to @hiiamboris for his deeply thought out <a href="https://github.com/red/red/wiki/[PROP]-Series-evolution">proposal regarding "series evolution,"</a> a framework for standardizing and testing the functions we use in Red for manipulating series. Design is hard, and we have a number of initiatives in the works taking a lot of brain power right now.<br /></li>
<li>Over 60 commits were made to <a href="https://github.com/red/red/tree/GTK">Red's GTK branch</a> in November, making it almost ready for "prime time." The product of a lot of work by a fair handful of the core team, heavy lifter @bitbegin says that merging to master branch is possibly the next step.<br /></li>
<li>And, to close, a bit of a tease: Watch this space for some snazzy website changes coming soon.</li>
</ul>
All the warmest wishes for the upcoming holidays, from the Red family to yours. <3<br />
<br />
-Lucinda.<br />
<br />Unknownnoreply@blogger.com0tag:blogger.com,1999:blog-5936111837781935054.post-45213022692432868602019-11-20T00:16:00.001+01:002019-11-21T20:17:16.872+01:00Editorial: A Brief Essay on Lexical Ambiguity by G. Irwin<i>The original commentary was posted in Red's Gitter channel, <a href="https://gitter.im/red/red?at=5dc32b09ef84ab37860a9b11">here</a>, by Gregg Irwin, one of our core team members, in response to various requests for the ability to create new datatypes in Red.</i><br />
<i><br /></i>
<i>As a writer, Red has always appealed to me because of its flexibility; but, of course, "the <a href="https://youtu.be/08MqYvU-yuM">[lexicon] devil</a> is in the details," as the idiom goes (okay, I edited that idiom a little, but it was too cool a link to pass up). It means the more specific we try to be, the more challenges and limitations we encounter, and we can lose some of the amazing versatility of the language. On the other hand, precision and refinement--the "exact right word at the exact right time," can powerfully enhance a language's utility. The dynamic tension between what he calls "generality and specificity, human friendliness and artifice," in the text below, can be an energetic ebb and flow that serves to strengthen our language, to make it more robust. </i><br />
<i><br /></i>
<i>Two quotes from community members provide some context:</i><br />
<div style="text-align: center;">
<i>_____________________________</i></div>
<br />
<div style="text-align: center;">
> The real problem is not number of datatypes, but the lexical syntax of the new ones. -@Oldes</div>
<div style="text-align: center;">
<br /></div>
<div style="text-align: center;">
> ...However if something like<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: lime;">utype!</span></span> is added, nothing prevents you from (ab)using <span style="color: lime; font-family: "courier new" , "courier" , monospace;">system/lexer/pre-load</span> and reinventing whole syntax. -@Rebolek</div>
<br />
<br />
"I don't support abusing <span style="color: lime; font-family: "courier new" , "courier" , monospace;">system/lexer/pre-load</span>, and (in the long view) there will almost certainly be special cases where a new lexical form makes sense. We can't see the future, so we can't rule it out. But, and this is key, how much value does each new one add?<br />
<br />
I believe that each new lexical form adds less value, and there is a point of diminishing returns. This is not just a lexical problem for Red, but for humans. We have limited capacity to remember rules, and a constrained hierarchy helps enormously here. Think more like linguists, and less like programmers or mathematicians.<br />
<br />
In language we have words and numbers. Numbers can be represented as words, with their notation being a handy shortcut for use in the domain of mathematics. And while we classify nouns, verbs, and adjectives by their use, they are all words, and don't have syntax specific to their particular part of speech. That's important because a single word may be used in more than one context, for more than one purpose.<br />
<br />
This is interesting, as a tangent, because human language can be ambiguous, though some synthetic languages try to eliminate that (e.g. Lojban). The funny thing is that it's almost impossible to write poetry or tell jokes in Lojban. Nobodyº speaks Lojban. This ties to programming because, while we all know the strengths and value of strict typing, and even more extreme features and designs meant to promote correctness, dynamic languages are used more at higher levels<i> [such as poetry, songwriting and humor, where even the sounds used in one single word can be employed to evoke specific emotive responses in the listener--the effects of devices like assonance, consonance, and loose associations we make with even single letters, in the way a repeated letter R throughout a line of poetry or literature can subtly impart a sense of momentum and intensity to the text...possibly because it evokes a growl... -Ed.]</i>. Why is that? Humans.<br />
<br />
When Carl designed Rebol, it had a goal, and a place in time. He had to choose just how far to go. Even what to call things like <span style="color: lime; font-family: "courier new" , "courier" , monospace;">email!</span>, which are very specific to a particular type of technology. This is what gives Redbol langs so much of their power. They were designed as a data format, meant for exchanging information. That's the core. What are the "things" we need to exchange information about with other humans, not just other programmers?<br />
<br />
Do I want new types? I'm pushing for at least one: <span style="color: lime; font-family: "courier new" , "courier" , monospace;">Ref!</span> with an <span style="color: red;">@some-name-here</span> syntax. It's not <span style="color: lime; font-family: "courier new" , "courier" , monospace;">username!</span> or <span style="color: lime; font-family: "courier new" , "courier" , monospace;">filename+line-number!</span>, or specific in any way. It's very general, as lexical types should be; their use and meaning being context-specific (the R in Redbol, which stands for "relative"). I also think<span style="color: lime; font-family: "courier new" , "courier" , monospace;"> ~ </span>could be a leading numeric sigil to denote approximation. It came mainly from wanting a syntax for floats, to make it clear that they are imprecise; but it's tricky, because it could also be much richer, and has to take variables into account. <span style="color: lime; font-family: "courier new" , "courier" , monospace;">~.1</span> is easy, but what about <span style="color: lime; font-family: "courier new" , "courier" , monospace;">x = ~n+/-5%</span>? Units are also high value, but they are just a combination of words and numbers. (Still maybe worth a lexical form.)<br />
<br />
When we look at what Red should support, and the best way to let users fulfill application and purpose-specific needs, we can learn from the past, and also see that there is no single right answer. Structs, Maps, Objects, data structures and functions versus OOP, strict vs dynamic.<br />
<br />
As Forth was all about "Build a vocabulary and write your program in that," think about what constitutes a vocabulary; a lexicon. It's a balance, in Red, between generality and specificity, human friendliness and artifice. So when we ask for things, myself and Nenad included, we should first try to answer our need with what is in Red today, and see where our proposed solution falls on the line of diminishing returns. To this end, we can and should abuse <span style="color: lime; font-family: "courier new" , "courier" , monospace;">system/lexer/pre-load</span> for experimentation."Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-5936111837781935054.post-62007696534800261692019-10-30T20:18:00.000+01:002019-11-07T07:45:31.188+01:00A Deeper Dive Into the Fast-Lexer Changes<h3 style="text-align: left;">
What made the fast-lexer branch a priority?</h3>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Several things. It started when <a href="https://github.com/dockimbel">@dockimbel</a> looked into ticket <a href="https://github.com/red/red/issues/3606">#3606</a>, which was impossible to fix currently, and we didn't want to give up on the auto-syncing between <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">/text</span> and <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">/data</span> facets. So he had to consider bigger options, including how to make the lexer instrumentable. It was not easy, because the current lexer is not re-entrant, so having the lexer emit events to a callback function could have caused serious problems.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Digging through all Red's repos showed that the current lexer code was duplicated twice, beyond the basic lexing needed by <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">load</span>: once in the console code, once in the VSCode plugin, each time for syntax coloring purposes, and each one lagging behind the original implementation. Not good.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<a href="https://github.com/dockimbel">@Dockimbel</a> then considered changing the current lexer to make it instrumentable, but the changes were significant and would have made the parse rules much more complex. At the same time, <a href="https://github.com/qtxie">@qtxie</a> did some benchmarking, and the result showed Red's lexer was ~240 times slower than Rebol's. This is not due to <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">parse</span>, but rather because the high-level rules were optimized for readability, not performance.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
The lexer also caused delays in the VSCode plugin, because of its (lack of) performance. The high level code has served Red well, and was a showcase for <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">parse</span>, but <span style="color: #93c47d; font-family: "courier new" , "courier" , monospace;">load</span>ing larger data is also being used by community members, and data sizes will just keep growing. With some projects we have on the horizon, the lexer's performance became a higher priority.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
As planned since the beginning (the lexer used to be R/S-only during the pre-Unicode era), <a href="https://github.com/dockimbel">@dockimbel</a> decided the best option was to not postpone the conversion of the lexer to pure R/S code any longer, by porting R3's C-based lexer to R/S. After studying Rebol's lexer in detail, he realized that the code was quite complex in some places (mostly the prescanner), and would lead to less than optimal R/S code that would be hard to maintain.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Evaluating the state of the art in fast parsers for programming languages, he found inspiration in some unpublished papers. He then started prototyping the next lexer in R/S, and realized that it could be several times faster than Rebol's, with the additional benefit of much smaller and simpler code. Then he embarked on the full implementation. Knowing he and <a href="https://github.com/qtxie">@qtxie</a> would not have the opportunity to work on that for probably a year with all the big tasks ahead on the roadmap, he committed to it full time.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
Red's new R/S lexer is half the size of Rebol's, far simpler, with more maintainable code, and it performs at similar speeds (sometimes a bit faster, sometimes a bit slower). That is a fantastic result, because it means that with an optimizing backend (Red/Pro), our lexer will be 4-8 times faster than R3's. It should then be possible to load gigabytes of Red data in memory in just a few</div>
<div style="text-align: left;">
seconds (using the future 64-bit version). 😉</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
An additional benefit was brought by <a href="https://github.com/qtxie">@qtxie</a>, who added a hashtable for symbol lookup in Red contexts. That sped up word loading tremendously, and should have a measurable improvement on Red's start up time; especially on slow platforms like Raspberry Pi.</div>
<div style="text-align: left;">
<br /></div>
<div style="text-align: left;">
<a href="https://github.com/dockimbel">@Dockimbel</a> is almost done with the lexer itself, just <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">date! </span><span style="font-family: inherit;">and</span><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;"> time!</span> to add, and it should be possible to replace the old one with the new one after thorough testing and debugging. Then, we'll add the hooks for a user-provided callback, allowing us to instrument the lexer in ways Redbolers could only dream about until now. One application of that will be the ability to implement "predictive loading," which will tell you the type and size of a Red value in a string, without loading it, and at extremely high speed (~370MB/s currently, 1-2GB/s with /Pro). Such a feature will allow us to finally address the <a href="https://github.com/red/red/issues/3606">#3606 issue</a> with a very clean and efficient solution, while keeping the facet's auto-syncing feature.</div>
<div style="text-align: left;">
<br /></div>
Unknownnoreply@blogger.com4tag:blogger.com,1999:blog-5936111837781935054.post-19724894811423101032019-10-25T08:14:00.000+02:002019-10-25T17:59:36.655+02:00October 2019 In ReviewOver the last few weeks the Red Lang core team drilled down to make some truly great progress on Red's fast-lexer branch--while we also gained valuable support from the contributions of Red doers and makers as they consolidate a world of useful information and resources.<br />
<b><br /></b>
<br />
<h3>
<span style="font-weight: normal;">Fast-Lexer Benchmarks</span></h3>
<br />
In the <a href="https://github.com/red/red/tree/fast-lexer">fast-lexer branch</a> of Red, you can see lots of new work from Red creator <a href="https://github.com/dockimbel">@dockimbel (Nenad Rakocevic</a>) and core teammate <a href="https://github.com/qtxie"> @qxtie</a>. Among other fixes and optimizations, they substituted a hashtable for what had previously been a large array in <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">context!</span><br />
<br />
The numbers so far: Loading 100'000 words (5 to 15 characters, 1MB file):
Red (master): 19000ms. Red (fast-lexer): 150ms. Nenad's observations on further testing:<br />
<blockquote class="tr_bq">
<i>"FYI, we just [ran] some simple benchmarks on the new low-level lexer for Red using 1M 10-digit integers. The new lexer completes the loading about 100 times faster than the current high-level one. Loading 1M 10-digit integers in one block: Red: 175ms; R2: 136ms; R3: 113ms. </i></blockquote>
<blockquote class="tr_bq">
<i>"We use a faster method than Rebol, relying on several lookup tables and a big FSM with pre-calculated transition table (while Rebol relies on a lot of code for scanning, with many branches, so bad for modern CPU with branch predictions). With an optimizing backend, Red's LOAD should in theory run 2-3 times faster than Rebol's one. (Though, we still need to optimize the symbol table loading in order to reach peak performance). Given that Rebol relies on optimized C code while Red relies on sub-optimal code from R/S compiler, that speaks volume about the efficiency of our own approach. So, Red/Pro should give us a much faster LOAD.</i></blockquote>
<blockquote class="tr_bq">
<i>"The lexer is not finished yet, but the hard part is done. We still need to figure out an efficient way to load keywords, like escaped character names (`^(line), ^(page), ...) and month nouns in dates." </i></blockquote>
This is a huge accomplishment, and it's shaping up to make future goals even more impressive. The <a href="https://github.com/red/red/tree/fast-lexer"> fast-lexer branch</a> is a work in progress, but stay tuned: Nenad has more to say about why it's been prioritized just now, which we will have in an upcoming post.<br />
<b><br /></b>
<br />
<h3>
<span style="font-weight: normal;">Red's MVPs Contribute New Resource Material & Tools</span></h3>
<br />
If you're new to Red, sometimes the flexibility of the language can leave you uncertain about which aggregate structure to use. In <a href="https://github.com/red/red/wiki">red/red's wiki</a> on github, <a href="https://github.com/9214">@9214</a> contributes a useful <a href="https://github.com/red/red/wiki/%5BDOC%5D-Comparison-of-aggregate-values-(block!-object!-hash!-map!)">guide</a> for those seeking to tease apart the differences. For example, <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">map!</span> works better with data that can be atomized, or framed as a conventional associative array, while <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">hash!</span> lends itself to data that will be queried at a high volume and which will require fewer updates. Learn further linguistic nuances, including <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">object!</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">block!</span>, as well as a useful comparison table of their algorithmic complexity, <a href="https://github.com/red/red/wiki/%5BDOC%5D-Comparison-of-aggregate-values-(block!-object!-hash!-map!)">here</a>. <a href="https://github.com/rebolek">@Rebolek</a>, meanwhile, has furnished us with loads of useful information, diving deeper into code statistics. His value datatype distribution, <a href="https://gitter.im/red/sandbox?at=5d94a2789d4cf17360577d04">here</a>, his unofficial Red build archive <a href="https://rebolek.com/builds/">here</a>, and his <a href="https://github.com/rebolek/red-tools">rebolek/red-tools repo</a> containing various tools--line parsers, codecs, APIs and documentation among them--are greatly appreciated. The tools repo has a number of new features you can check out <a href="https://gitter.im/red/red?at=5daa09d93e33a9652af1c123">here</a>.<br />
<br />
<br />
<h3>
<span style="font-weight: normal;">About Those Ports...</span></h3>
<br />
Wondering about <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">port!</span>? <a href="https://gitter.im/red/red?at=5d94c8289d4cf1736058b0a4">Here's the latest</a>. We've got <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">port!</span> in the master branch already, but low-level input/output networking abilities aren't complete yet, so we need to focus on this, and your feedback can always help. "We have a working async TCP and TLS ports implementation (both client and server-side)," explains Nenad, "but they still require more work to cover all the target platforms." <a href="https://gitter.im/red/red?at=5d94c8289d4cf1736058b0a4">Here</a>, he goes on to explain the prerequisites for our team to complete this process; your thoughts and code contributions are welcomed.<br />
<br />
<br />
<h3>
<span style="font-weight: normal;">Games and Experiments</span></h3>
<br />
It's <a href="https://gitter.im/red/red/gui-branch?at=5d91f526fcb47b627fefd967">a fun one</a> to end this update on: Red community member <a href="https://github.com/GalenIvanov">@GalenIvanov</a>'s "Island Alleys," a game of unspooling Hamiltonian paths! A path of this type only allows its line, which inscribes a closed loop, to cross through a vertex within a graph once, a process which can lend itself to neural network-related interpretations. And <a href="https://gitter.im/red/red/gui-branch?at=5d91f526fcb47b627fefd967">@planetsizedcpu</a> offers a wintry little spin on <a href="https://github.com/planetsizecpu/Cave-In">this repo</a>. Enjoy, and thanks to all!Unknownnoreply@blogger.com5tag:blogger.com,1999:blog-5936111837781935054.post-89029725802131194142019-09-15T21:23:00.000+02:002019-09-15T21:56:45.646+02:00The Latest: Red could help AI be more precise; community stars; one CSV codec to rule them all?Hello to all the great makers, doers and creative people who are using Red, helping the Red Language grow and improve!
As always, there's a standing invitation for you to join us on <a href="https://gitter.im/red/help" target="_blank">Gitter</a>, <a href="https://t.me/redofficial" target="_blank">Telegram</a> or <a href="https://github.com/red" target="_blank">Github</a> (if you haven't already) to ask questions and tell us about your Red-powered projects.<br />
<br />
Here are some recent highlights we’d like to share with you:<br />
<br />
<h4>
1. Tickets Get Priority</h4>
In the last month, our core team has closed a large number of tickets.We’d like to thank community members <a href="https://github.com/red/red/issues/4006" target="_blank">rgchris</a>, <a href="https://github.com/red/red/issues/3950" target="_blank">giesse</a>, and <a href="https://github.com/red/red/issues/3915" target="_blank">dumblob</a> who are just a few of the passionate contributors putting Red through its paces and providing feedback as fixes and changes occur. @WArP ran the numbers for us, showing a cyclical growth pattern linking bursts of closed issues and some serious Red progress, and September’s not even done yet!...:<br />
<img height="246" src="https://lh4.googleusercontent.com/YDTKrNALQF7Q_z27RQE1Z9CYPT4rihiZaRcoFG55Jw4MlOVps7NlIH14sVHbns5Qhrq_Khx1UyJq7NshSk9-sZYods6RgfVWUZAEacC7b1oSYU6lqnAz9wFDO-SjDXfU1XECdl4T" style="background-color: white; color: #222222; font-family: arial; font-size: 11pt; margin-left: 0px; margin-top: 0px; white-space: pre-wrap;" title="Chart" width="398" /><br />
<br />
<h4>
2. CSV Codec Available</h4>
Our newly updated <a href="https://github.com/red/red/wiki/CSV-codec" target="_blank">CSV codec</a> has been merged in the master branch and is now a part of the nightly (or automatic) build <a href="https://www.red-lang.org/p/download.html" target="_blank">here</a>. It is in an experimental phase, and we want your <a href="https://gitter.im/red/red" target="_blank">feedback</a>.<br />
<br />
Should the standard codec only support block results, so it’s as simple as possible? Or do people want and need record and column formats as well (using the <span style="color: #38761d; font-family: "courier new" , "courier" , monospace;">load-csv/to-csv</span> helper funcs, rather than <span style="color: #38761d; font-family: "courier new" , "courier" , monospace;">load/as</span>)? Including those features as standard means they’re always available, rather than moving them to an extended CSV module; but the downside is added size to the base Red binary.<br />
<br />
Applause goes to @rebolek’s excellent organization and his <a href="https://github.com/red/red/wiki/CSV-codec" target="_blank">wiki</a> on the codec, which explains the various ways in which Red can represent data matrices. He writes, “Choosing the right format depends on a lot of circumstances, for example, memory usage - column store is more efficient if you have more rows than columns. The bigger the difference, the more efficient.”<br />
<br />
You can judge their efficiency <a href="https://gist.github.com/rebolek/e9c718175a0c60c1ec1a6e1a97d8cd2c" target="_blank">here</a>, where @rebolek has laid out the compile time, size and speed of each version, including encapping and lite. Be sure to get the <a href="https://www.red-lang.org/p/download.html" target="_blank">latest build</a>, and chat with everyone on <a href="https://gitter.im/red/red" target="_blank">Gitter</a> to tell us what you think.
<br />
<br />
<h4>
3. Red has reached <a href="https://github.com/red/red/stargazers" target="_blank">4K stars</a> on GitHub!</h4>
We’re truly grateful for all the interest and support, and we are proud of the way our growth has been powered by this community.<br />
<br />
<h4>
4. AI + Red Lang Stack: Precision Tuning With Local OR Web-Based Datasets</h4>
In conversation with @ameridroid:<br />
“Presently, it seems like most AI systems available today either allow building an AI from scratch using low level code (difficult and time-consuming), *OR* using a pre-built AI system that doesn't allow any fine-tuning or low-level programming...with the advent of NPUs (Neural Processing Units) akin to CPUs and GPUs, an AI toolkit would allow specifying what type of AI we want to perform (facial, image or speech recognition, generic neural net for specialized AI functions, etc.), the training data (images, audio, etc.) and then allow us to send it the input data stream and receive an output data stream…[using Red] would also allow us to integrate with the AI system at a low level if we have specific needs not addressed by the higher-level funcs. Red dialects would be a good way to define the AI functionality that's desired (a lot like VID does for graphics), but also allow the AI components, like the learning dataset or output data stream sanitization routines, to be fine-tuned via functions. Red can already work on web-based data using 'read or 'load, or work on local data in the same way; the learning data for a particular AI task could be located on the web or on the local machine. That's not easily possible with a lot of the AI solutions available today.”<br />
<br />
Check back in the next few days for an update from @dockimbel!<br />
<br />
Ideas, contributions, feedback? Leave a comment here, or c’mon over and join our conversation on <a href="https://t.me/redofficial" target="_blank">Telegram</a>, <a href="https://gitter.im/red/help" target="_blank">Gitter</a>, or <a href="https://github.com/red" target="_blank">Github</a>.Unknownnoreply@blogger.com9tag:blogger.com,1999:blog-5936111837781935054.post-28301287906360399442019-07-18T18:08:00.000+02:002019-07-25T18:06:03.534+02:00GPIO port for Raspberry PiAs part of the R&D work on port! datatype in port-type branch, we have implemented a <a href="https://en.wikipedia.org/wiki/General-purpose_input/output" target="_blank">GPIO</a> driver for <a href="https://www.raspberrypi.org/" target="_blank">Raspberry Pi</a> boards, as a <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">gpio://</span> scheme (no third-party library needed). This work helped define the low-level API for ports written in purely Red/System or a mix of Red and Red/System.<br />
<br />
The Raspberry Pi is a very popular board with millions of units sold, so this is a market where Red could be potentially helpful to developers. We could run Red on such boards for years but did not have proper GPIO support, so this is now fixed!<br />
<br />
The current features supported by the GPIO port are:<br />
<ul>
<li>auto-detecting the Raspberry Pi board type</li>
<li>uses <span style="font-family: "courier new" , "courier" , monospace;">/dev/mem</span> or<span style="font-family: "courier new" , "courier" , monospace;">/dev/gpiomem</span> for direct and fast access.</li>
<li>reading a GPIO pin state.</li>
<li>writing to a GPIO pin state.</li>
<li>hardware PWM output support (on capable pins).</li>
<li>a simple DSL for sending commands.</li>
</ul>
<br />
Planned (but not scheduled) future features include:<br />
<ul>
<li>generating events when a pin state changes.</li>
<li>software PWM on all GPIO pins.</li>
<li>a higher-level reactive object layer for a API-less interface.</li>
<li>various drivers for common peripherals.</li>
</ul>
<br />
The source code for the <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">gpio://</span> scheme can be found <a href="https://github.com/red/red/blob/master/environment/schemes/GPIO.red" target="_blank">there</a>.<br />
<br />
In the short video below, you can find a little pet project meant for testing some of the features. It implements a simple joypad with 4 directions support, a red LED indicating when the pad is active and a green one for notifying when a level is completed. The game is the <a href="https://github.com/hyzwhu/redbox" target="_blank">Red port</a> contributed by Huang Yongzhao of <a href="https://www.softinnov.org/rebol/rebox.shtml" target="_blank">Rebox!</a>, my old clone of BoxWorld written in Rebol. In the video below, it is running on a Raspberry Pi 3 using our work-in-progress <a href="https://github.com/red/red/tree/GTK" target="_blank">red/GTK</a> backend for Red (contributed by Rcqls), locally merged with the red/port-type development branch (EDIT: that branch has been merged into master now).<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/Gqc1nz61PtU/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/Gqc1nz61PtU?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
You can find the breadboard layout below made using <a href="http://fritzing.org/" target="_blank">Fritzing</a>. Sorry for the messy wiring, it is my first try with such kind of tool. If you have the skills to improve it, here is the <a href="https://static.red-lang.org/sketches/RPi3-joypad.fzz" target="_blank">sketch file</a>. The buttons rely on the internal pull-down resistors. Note that in the video, the board is reversed.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhamoIRQ5bvTpzSpJJaIAY7iQ7mt7Ybtbi-YzpqDM00lt9spyzgoO6kQOKmINVPZe27cXR32kkoZIgHtS7vTo6pdyhRsmjLl3HPWCoChnw09lqChnke8JElY3Itz8gDxZ9HWKF2no42L1I-/s1600/RPi3-joypad_bb.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1377" data-original-width="1080" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhamoIRQ5bvTpzSpJJaIAY7iQ7mt7Ybtbi-YzpqDM00lt9spyzgoO6kQOKmINVPZe27cXR32kkoZIgHtS7vTo6pdyhRsmjLl3HPWCoChnw09lqChnke8JElY3Itz8gDxZ9HWKF2no42L1I-/s400/RPi3-joypad_bb.png" width="311" /></a></div>
<br />
The source code of Redbox has been modified to add GPIO support, you can find the modified code <a href="https://gist.github.com/dockimbel/46b358bdda64f60c7f7aaa9e2bc32033" target="_blank">here</a>. The GPIO-related code is enclosed in a context:<br />
<br />
<pre class="code"> joypad: context [
mapping: [
16 down 20 up
17 left 21 right
]
row: [(id) state: #[false] direction: (direction)]
table: collect [
foreach [id direction] mapping [keep compose row]
]
pins: extract table length? row
port: none
acquire: has [pin][
port: open gpio://
foreach pin pins [
insert port [
set-mode pin in
pull-down pin
]
]
insert port [
set-mode 18 out
set-mode 4 out
set 4 on
]
]
pressed?: function [][
foreach pin pins [
entry: find table pin
old: entry/state
insert port [get pin]
entry/state: make logic! port/data
;-- detect 0-to-1 transitions only, to avoid auto-firing
if all [not old entry/state][return entry/direction]
]
none
]
show-win: does [insert port [set 18 on]]
release: does [
insert port [
set 4 off
set 18 off
]
close port
]
]
</pre>
<br />
The GPIO port has a simple API:<br />
<br />
<ul>
<li>opening: <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">port: open gpio://</span></li>
<li>sending commands: <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">insert port [...commands...]</span></li>
<li>closing: <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">close port</span></li>
</ul>
<br />
The sent commands form a small DSL:<br />
<pre class="code">Set the working mode for a given pin:
set-mode <pin> <mode>
<pin> : pin number (integer!)
<mode>: in, out, pwm
Write a value on a pin:
set <pin> <value>
<pin> : pin number (integer!)
<value>: true, false, on, off, yes, no, 0, 1
Read a value from a pin (the returned value is in port/data):
get <pin>
<pin>: pin number (integer!)
Manage pull-up/down resistors:
pull-off <pin> ;-- disable any pull previously set
pull-down <pin> ;-- activate pull-down on the given pin
pull-up <pin> ;-- activate pull-up on the given pin
Write a PWM value on a pin:
set-pwm <pin> <value>
<pin> : pin number (integer!)
<value>: an integer between 0 and 1024, or a percentage for duty cycle.
Fade in/out values on a PWM pin:
fade <pin> from <start> to <end> <delay>
<pin> : pin number (integer!)
<start>: starting value (0-1024)
<end> : ending value (0-1024)
<delay>: duration of the whole fading (time!)
Wait for a given duration:
pause <delay>
<delay>: integer => pause in miliseconds, float => pause in seconds.
</pre>
<br />
<u>Important notes</u><br />
<ul>
<li>Only BCM numbering for GPIO pins is supported.</li>
<li>A word, or path or paren expression containing regular Red code can be used in place of any numeric value.</li>
<li>A single command block can contain an arbitrary number of commands.</li>
<li>The <span style="color: #6aa84f;"><span style="font-family: "courier new" , "courier" , monospace;">get</span> </span>command can be used multiple times, a block of corresponding results will then be returned in <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">port/data</span>.</li>
<li>Use <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">sudo</span> when running your GPIO code if it involves PWM!</li>
<li>An example of using <span style="color: #6aa84f;"><span style="font-family: "courier new" , "courier" , monospace;">fade</span> </span>and other commands is available <a href="https://github.com/red/red/blob/master/environment/schemes/GPIO.red#L511" target="_blank">here</a>.</li>
<li>The PWM range is currently preset to 1024, though, the DSL can be trivially extended to allow a user-provided range value.</li>
</ul>
<br />
Happy hardware hacking! ;-)<br />
<br />Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com0tag:blogger.com,1999:blog-5936111837781935054.post-16447697886192168242019-07-09T20:12:00.000+02:002019-07-09T21:07:12.044+02:00RED Wallet 0.4.0We are releasing today the version 0.4.0 of the RED Wallet with several major new features. As a reminder, the RED Wallet aims to be a simple and very secure wallet for the major cryptocurrencies (BTC, ETH, and ERC-20 tokens). Safety is enforced through the mandatory use of a hardware key (Ledger or Trezor keys), which protects against any failure of the wallet app (being it a bug or an attacker). The RED Wallet code is fully written in Red (using the <a href="https://static.red-lang.org/red-system-specs-light.html" target="_blank">Red/System DSL</a> for USB drivers implementation) and <a href="https://github.com/red/wallet" target="_blank">open source</a>.<br />
<br />
<span style="font-size: large;">BTC support</span><br />
<br />
The RED Wallet now supports the bitcoin network. The main features are:<br />
<ul>
<li>retrieval of address balances.</li>
<li>sending transactions (with follow-up on an online block explorer).</li>
<li>support both SegWit and Legacy addresses.</li>
<li>support both mainnet and testnet.</li>
</ul>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVgMIEp2opCJC2kLx7SM5af-Ly3Ntm14KHyHIVZ6eSOQTOQuV7Iz9LhT6uAxuJSwpT0uH4jvDcUetUiYjYupA9_7E3gIyzt2MRL_Y17K8oyAh6O3IJnOJOzUzEmbxHDoHJuy-0g5zHjT2K/s1600/wallet040.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="400" data-original-width="908" height="175" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiVgMIEp2opCJC2kLx7SM5af-Ly3Ntm14KHyHIVZ6eSOQTOQuV7Iz9LhT6uAxuJSwpT0uH4jvDcUetUiYjYupA9_7E3gIyzt2MRL_Y17K8oyAh6O3IJnOJOzUzEmbxHDoHJuy-0g5zHjT2K/s400/wallet040.png" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5kkQ3c3Y_ehxcb0jOjlq7XEH8HgZgF-hlYjkosaalk6NQ-3BGzsTg6gzrS2z1Is9k-Y_R_frcb0mV7DLFGSsZChDRGM4xCIjEKTfrNr0uiK3r0imJ1AGW8TnFLrv3CycpVZRhfDaA_mgb/s1600/wallet040-4.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="752" data-original-width="946" height="317" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5kkQ3c3Y_ehxcb0jOjlq7XEH8HgZgF-hlYjkosaalk6NQ-3BGzsTg6gzrS2z1Is9k-Y_R_frcb0mV7DLFGSsZChDRGM4xCIjEKTfrNr0uiK3r0imJ1AGW8TnFLrv3CycpVZRhfDaA_mgb/s400/wallet040-4.png" width="400" /></a></div>
<br />
The balance retrieval can be quite slow on the bitcoin network, so be patient.<br />
<br />
<br />
<span style="font-size: large;">ERC-20 tokens support</span><br />
<span style="font-size: large;"><br /></span>
ERC-20 tokens are supported since version 0.3.0. Though, Ledger removed the ERC-20 tokens list from the embedded Ethereum app since version 1.2.6, leaving it to the wallet app to manage the tokens list. This release now incorporates that list in the RED Wallet binary directly, resulting in a significantly bigger binary (about 100 KB bigger than 0.3.0). The Nano keys will still check the validity of the tokens addresses internally for extra safety. Another consequence of Ledger's recent firmware updates is noticeably slower addresses retrieval when plugging the key. That's an internal change in the Ledger key, so we cannot do anything to bring back the fast speed we could achieve with older firmware.<br />
<br />
As a reminder, BIP44 address derivation support is present in the wallet since 0.3.0. If you are using a key initialized in an older Ledger Live version, the derivation path would be different and in order to make the wallet recognize your key, you need to click on the "(Legacy)" label (we will improve that UI in further versions).<br />
<br />
Batch transactions were introduced in 0.2.0, they now work for any ERC-20 tokens in addition to ETH. They can be accessed from a contextual menu using a right click on an ETH account.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglVNC2wu4Nl9J4QnpDInvQFPRXGeGFO1fDIpA8LlF1uuTszZ0GK4jx4W8RdQZapCQSNtfM4o7wyXDg54Ca82W8zSdkguZOyCHrkGmLlKBxYIpEWGaEW2-wWnYvXwXGo-7gVa1dJFfk1vmO/s1600/wallet040-3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="763" data-original-width="1238" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglVNC2wu4Nl9J4QnpDInvQFPRXGeGFO1fDIpA8LlF1uuTszZ0GK4jx4W8RdQZapCQSNtfM4o7wyXDg54Ca82W8zSdkguZOyCHrkGmLlKBxYIpEWGaEW2-wWnYvXwXGo-7gVa1dJFfk1vmO/s400/wallet040-3.png" width="400" /></a></div>
<br />
<br />
<span style="font-size: large;">TREZOR Key</span><br />
<br />
The RED Wallet has updated the firmware support:<br />
<br />
<ul>
<li>Wallet v0.3.0: Compatibility with firmware 1.7.1+</li>
<li>Wallet v0.4.0: Support for the latest firmware of the Trezor key. 1.8</li>
</ul>
<br />
TREZOR model T is fully supported on macOS. On Windows 7, if the key is not recognized, you need to install the <a href="https://wallet.trezor.io/#/" target="_blank">TREZOR bridge</a> first.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga0fh2vHieEU75t9pLa2tZiEOMzjy-6kgE4Xgzf85xTaB-SZSGts0EOmRyjvgRJjEtdWTjeTZyOCc_u3YgK_zR-jbUy4trARfP_tqriRZzlspDKiO9CoJhosDAm8-WMYLhi7xxl9LOdUUP/s1600/wallet040-5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="551" data-original-width="968" height="227" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEga0fh2vHieEU75t9pLa2tZiEOMzjy-6kgE4Xgzf85xTaB-SZSGts0EOmRyjvgRJjEtdWTjeTZyOCc_u3YgK_zR-jbUy4trARfP_tqriRZzlspDKiO9CoJhosDAm8-WMYLhi7xxl9LOdUUP/s400/wallet040-5.png" width="400" /></a></div>
<br />
<span style="font-size: large;">Bug fixes</span><br />
<br />
<ul>
<li>FIX: Ledger key does not work on Windows in some cases.</li>
<li>FIX: crashes in some cases when using Ledger key on Windows.</li>
<li>FIX: only ledger key is usable when plugging both ledger key and trezor key.</li>
<li>FIX: trezor-driver context name is wrong.</li>
</ul>
<br />
<br />
<span style="font-size: large;">Download the RED Wallet 0.4.0</span><br />
<span style="font-size: large;"><br /></span>
<br />
<div style="text-align: center;">
<a class="dl-button" href="https://static.red-lang.org/wallet/dl/win/RED-Wallet.exe" style="color: white;">Windows 7/8/10 <el style="color: black;"> [494 KB]</el></a>
<a class="dl-button" href="https://static.red-lang.org/wallet/dl/mac/RED-Wallet.zip" style="color: white;">macOS 10.1x <el style="color: black;"> [491 KB]</el></a>
</div>
<br />
<br />
Just click on the executable to run it (extract the .app file first on macOS), no installation or setup process required. Plug in your Ledger or TREZOR key and enjoy a smooth and simple experience!<br />
<br />
If you want to check if your RED Wallet binary has been tampered with in any way, you can simply drag'n drop the wallet executable on our <a href="https://red.github.io/bincheck/" target="_blank">binary checking service</a>. If it's legit, the screen will turn green. If it turns red with a warning message, please notify us on <a href="https://gitter.im/red/blockchain" target="_blank">Gitter</a> or <a href="https://twitter.com/red_lang" target="_blank">Twitter</a> at once.<br />
<br />
<u>Only download the RED Wallet app from this page, do not trust any other website for that</u>.<br />
<br />
Enjoy!<br />
<br />Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com2tag:blogger.com,1999:blog-5936111837781935054.post-17113737065376677852019-06-30T16:44:00.001+02:002022-01-22T17:36:13.167+01:00June 2019 UpdateIf you are an active member of the Red community, you will have likely been keeping track of Red's progress through our Gitter rooms and Github repositories, but we wanted to officially bring everyone up to speed with a shiny new blog post. The team has been busy since January on many fronts, multiplexing its efforts on many different branches of development. Here is an overview of what we have been cooking and what is coming.<br />
<br />
<span style="font-size: large;">RED Wallet</span><br />
<br />
We are preparing to release v0.4.0 of the <a href="https://www.red-lang.org/2018/06/red-wallet-tiny-simple-fast-and-secure.html" target="_blank">RED Wallet</a> which will feature:<br />
<ul>
<li>Bitcoin support with both Legacy and SegWit addresses!</li>
<li>Updated support for ERC-20 tokens for Ledger Nano keys (after Ledger externalized such token management to the wallet software earlier this year).</li>
<li>Support for the latest firmware of the Trezor key.</li>
<li>Some bug fixes.</li>
</ul>
The new wallet is currently under heavy testing, we plan to release it publicly in about a week.<br />
<br />
<span style="font-size: large;">C3</span><br />
<br />
The first public alpha of the C3/System compiler is coming this summer. It is still under heavy development though. C3/High has made little progress as it needs real-world use-cases from the smart contract ecosystem for modelling the DSL(s). That ecosystem has not spread around much, probably due to the so-called "crypto-winter". As new big players are now entering the space, we are looking forward to the possibilities for providing new backends for the C3/System, like for the <a href="https://libra.org/en-US/open-source-developers/" target="_blank">Libra</a> network.<br />
<br />
We will be soon releasing our ETH connector for Red. The final part missing is the integration of hardware and software key support for signing transactions on real online networks (we use private networks during development). The code for hardware key support is already there in our wallet, so we just need to integrate it inside the connector. For software keys, we have developed specific <a href="https://github.com/bitbegin/eth-wallet" target="_blank">support</a> for them, but it relies on third-party libraries, which are currently not a good match for our single binary distribution model.<br />
<br />
<span style="font-size: large;">GTK support</span><br />
<br />
Red should offer soon a pretty good GUI backend for Linux, thanks to the community effort led by <a href="https://github.com/rcqls" target="_blank">R cqls</a>. The <a href="https://github.com/red/red/pull/3928" target="_blank">last PR</a> for red/GTK branch weighs in at about 400 commits and covers most of View features.<br />
<br />
The main remaining features to cover are:<br />
<ul>
<li>Camera widget support (already available in an experimental branch)</li>
<li>Rich-text and Draw improvements</li>
<li>Shape dialect support</li>
<li>Some stability fixes</li>
</ul>
<div>
<br />
Here are some examples of scripts and apps using the GTK backend:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://files.gitter.im/red/GTK/TGpt/Capture-d_ecran-2019-04-28-a-22.18.03.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img data-original-height="772" data-original-width="800" height="192" src="https://files.gitter.im/red/GTK/TGpt/Capture-d_ecran-2019-04-28-a-22.18.03.png" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="200" /></a><a href="https://files.gitter.im/red/GTK/E9v6/Capture-d_ecran-2019-03-31-a-19.44.04.png" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="800" data-original-width="537" height="200" src="https://files.gitter.im/red/GTK/E9v6/Capture-d_ecran-2019-03-31-a-19.44.04.png" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="133" /></a><a href="https://files.gitter.im/red/GTK/QbWn/Capture-d_ecran-2019-04-20-a-17.23.42.png" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="800" data-original-width="666" height="200" src="https://files.gitter.im/red/GTK/QbWn/Capture-d_ecran-2019-04-20-a-17.23.42.png" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="166" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
GTK theming is also supported:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://files.gitter.im/red/GTK/n7t0/Capture-d_ecran-2019-04-25-a-13.43.27.png" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="572" data-original-width="800" height="227" src="https://files.gitter.im/red/GTK/n7t0/Capture-d_ecran-2019-04-25-a-13.43.27.png" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://files.gitter.im/red/GTK/vUQT/Capture-d_ecran-2019-04-25-a-13.38.50.png" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="572" data-original-width="800" height="227" src="https://files.gitter.im/red/GTK/vUQT/Capture-d_ecran-2019-04-25-a-13.38.50.png" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="320" /></a></div>
<div>
<br /></div>
<span style="font-size: large;">Red/System</span><br />
<span style="font-size: medium;"><br /></span>
The ARM backend got many improvements:<br />
<ul>
<li>Largely improved ARMhf (hard-float) support.</li>
<li>Fixes on regressions for ARMsf (soft-float).</li>
<li>Various libraries loading fixes, now all unit tests are passing again.</li>
<li>New ARMhf target added to our <a href="https://travis-ci.org/red/red" target="_blank">Travis CI</a> backend.</li>
</ul>
<br />
A few low-level features were <a href="https://static.red-lang.org/red-system-specs-light.html#section-20" target="_blank">added</a>, that would make it easier to write device drivers or operating systems directly in Red/System:<br />
<br />
<ul>
<li>Ability to save/reload all CPU registers on stack using intrinsics.</li>
<li>Ability to read/write CPU I/O ports using new intrinsics.</li>
<li>Ability to inline machine-code pretty much anywhere in R/S source code.</li>
<li>Replacing hard native support for modulo operation on floats by the soft <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">fmod()</span> import.</li>
</ul>
<div>
<br /></div>
<div>
<span style="font-size: large;">VSCode plugin</span></div>
<div>
<br /></div>
<div>
The <a href="https://marketplace.visualstudio.com/items?itemName=red-auto.red" target="_blank">Red plugin</a> for VSCode got a major refresh with a full <a href="https://github.com/red/VScode-extension" target="_blank">reimplementation</a> using <a href="https://langserver.org/" target="_blank">LSP model</a> improving the efficiency of existing features and adding some new ones:</div>
<div>
<ul>
<li>Smarter code completion</li>
<li>Realtime syntax checking</li>
<li>Goto definition support</li>
</ul>
<div>
<br /></div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://raw.githubusercontent.com/red/VScode-extension/master/images/goto-definition.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="677" data-original-width="800" height="168" src="https://raw.githubusercontent.com/red/VScode-extension/master/images/goto-definition.gif" width="200" /></a><a href="https://raw.githubusercontent.com/red/VScode-extension/master/images/completion.gif" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="677" data-original-width="800" height="168" src="https://raw.githubusercontent.com/red/VScode-extension/master/images/completion.gif" width="200" /></a></div>
<div>
<br /></div>
<br />
<span style="font-size: large;">Other WIP features</span><br />
<span style="font-size: large;"><br /></span>
<b>JSON support</b><br />
<br />
JSON format is now <a href="https://github.com/red/red/blob/master/environment/codecs/JSON.red" target="_blank">supported</a> by Red's codec system. So it can be used in <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">load/as</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">save/as</span> to convert JSON data back and forth to Red values. More info about its usage in the Red 0.6.5 release notes.<br />
<br />
<b>Shadows in Draw</b><br />
<br />
Some simple shadow effects in Draw have been <a href="https://github.com/bitbegin/red/commits/blur-demo" target="_blank">implemented</a>. They have the capabilities of the <a href="https://developer.mozilla.org/en-US/docs/Web/CSS/box-shadow" target="_blank">box-shadow</a> model of CSS.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZhc-tLvwcS25Mh_pWQwCFIFQEZJYCSAAdAtkP2ADtDQv5npjJwwzrmQpPdYVXb-WwRQ9JPEiu6okX25eEggjnwwFAnpMoEWQH2xzfQnj5dQeNTWc5cERG_YPIadyYYf-p9O8CkTYFQjhe/s1600/photo_2019-06-30_22-46-26.jpg" style="margin-left: 1em; margin-right: 1em;"><img data-original-height="593" data-original-width="1062" height="222" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhZhc-tLvwcS25Mh_pWQwCFIFQEZJYCSAAdAtkP2ADtDQv5npjJwwzrmQpPdYVXb-WwRQ9JPEiu6okX25eEggjnwwFAnpMoEWQH2xzfQnj5dQeNTWc5cERG_YPIadyYYf-p9O8CkTYFQjhe/s400/photo_2019-06-30_22-46-26.jpg" style="-moz-box-shadow: none; -webkit-box-shadow: none; background: none; border: 0px; box-shadow: none; padding: 0px;" width="400" /></a></div>
<br />
<br />
<b>GPIO library for Raspberry Pi</b><br />
<br />
Red has supported Raspberry Pi as a compilation target for many years, but we were still lacking a proper GPIO library, which is now under development. It will be implemented as a <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">gpio://</span> port for now in the <span style="colour: #6aa84f; font-family: "courier new" , "courier" , monospace;">port-type</span> branch. A dedicated library or dialect should be added also as a higher abstraction layer in order to simplify GPIO programming even further. Hobbyists and seasoned programmers will be able soon to build IoT projects on RPi easily using Red with GTK for the GUI support.<br />
<br />
<br />
<span style="font-size: large;">Next Red releases</span><br />
<br />
<b>0.6.5: Splitting the console and compiler</b><br />
<br />
This is an important change that will be the new way to use Red and, hopefully, provide an even better experience, especially for newcomers. Not much coding is involved but we still need a prototype before giving the final go-ahead then proceeding with the code and infrastructure changes. We plan to finish those tasks during this summer (no precise ETA for now).<br />
<br />
<b>0.7.0: Full async I/O</b><br />
<br />
Work started on this release many months ago but with a low-priority, as we have many other features to release first. The current state on the main features is:<br />
<ul>
<li>port! type is almost completed and already usable (e.g. the eth:// port) </li>
<li>the native ports TCP, UDP, DNS should be implemented this summer.</li>
<li>the async API still needs design work before being implemented.</li>
</ul>
<div>
<br /></div>
<div>
<span style="font-size: large;">Tokens retro-distribution</span></div>
<div>
<br /></div>
<div>
We are setting up the last steps for the RED tokens retro-distribution (past contributions) and the first monthly distribution for new contributors. We will publish the first article in July that will provide all the rules that we have established for that process. Then shortly after, we will publish the list of the contributors who were selected according to those rules, with the reward amounts and proceed with the distribution (using the batch transactions feature of the RED Wallet).</div>
<br />
For example, as part of our tokens rewarding program for contributors, R cqls should receive a good amount of them in the next distribution,<br />
<br />
<br />
<span style="font-size: large;">The road ahead...</span><br />
<br />
The most notable next planned milestones are:<br />
<ul>
<li>RED Wallet 0.4.0 in July.</li>
<li>C3/System alpha release in August!</li>
<li>Red/Android beta release in September!</li>
<li>Red/Pro for New year!</li>
<li>Red 64-bit will be split into two phases:</li>
<ul>
<li>cross-compiled from 32-bit platform: end of the year</li>
<li>toolchain ported to 64-bit: 2020</li>
</ul>
</ul>
<br />
We plan to give more information about each of those big new additions to the Red family this summer.<br />
<br />
As several development tracks are coming to fruition in the next couple of months, new articles should come up more frequently.<br />
<br />
Go Red Go!Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com8tag:blogger.com,1999:blog-5936111837781935054.post-89779531441051889192019-02-14T00:10:00.000+01:002019-02-16T08:42:35.887+01:00January 2019 Update<br />
<br />
<h2>
Stepping-up</h2>
<br />
2019 started with continued advances in the main development branch, implementation of features, and triage of pending tickets. For example, smart merging of a style's actors with the actors of a particular face instance was introduced in the <a href="https://www.red-lang.org/2016/03/060-red-gui-system.html">VID dialect</a>. Example:<br />
<br />
<pre><code>view [
style my-button: base
on-down [face/color: face/color / 2 do-actor face event 'click]
on-up [face/color: face/color * 2]
my-button "Say hi!" red on-click [print "hi"]
]
</code></pre>
<br />
But this is just a tiny drop in the January ocean, and nothing special in the grand scheme of things. More than 50 commits to the mainline, lots of doc updates, and many improvements to the Red Wallet, which we'll talk about in another blog post. On the other hand, Red's progress is rarely trivial and silky smooth, and <a href="https://github.com/red/red/issues/3692#issuecomment-450722963">this write-up</a> gives a deeper insight into our daily routine. Well, what did you expect, Pumpkin Juice? ;-)<br />
<br />
Alas, as much as we love coding, organizational heavy-lifting needs to be done too — one of our short-term priorities this year is relocation of core team to Europe, and (ongoing) activity on this front took a lot of our time and resources in January. That is to say, after little overture, the core team's main activity shifted behind the scenes, giving the stage to our outstanding community's efforts. <sup><a href="https://www.red-lang.org/2019/02/january-2019-update.html#footnote-1">1</a></sup><br />
<br />
<h2>
</h2>
<h2>
</h2>
<h2>
Community highlights</h2>
<h2>
</h2>
<h3>
GTK backend</h3>
<br />
Red strives to be a truly cross-platform language, but the <a href="https://trello.com/b/FlQ6pzdB/red-tasks-overview">project's roadmap</a> always implies a certain order of priorities, where some milestones come later than others. One such missing cross-platform feature is the View engine's backend for Linux — its development requires dedicated resources, which the core team cannot fully afford, probably until near the 1.0 release.<br />
<br />
With that in mind, it is much to our delight that <a href="https://github.com/red/red/pull/3718">development of GTK backend</a> is being carried out as a dedicated community effort. We'd like to take an opportunity and thank all the people participating in it — their proactive contributions make Red a truly open-source project, and help us to advance it towards our collective vision. Big thanks to <a href="https://github.com/rcqls">@rcqls</a> for taking the lead on this.<br />
<br />
Last but not least, you can track development's progress in a recently created <a href="https://gitter.im/red/GTK">GTK room</a>. Don't forget to visit it and give our contributors all the praise they deserve! Or even better — become one of them and get a share of fame and glory! ;-)<br />
<br />
<br />
<h3>
</h3>
<h3>
</h3>
<h3>
View and Rich-text tweaking</h3>
<br />
Not only do missing parts of the View engine get developed, but existing ones receive thorough maintenance too. Features <i>and</i> fixes.<br />
<br />
Among fixes are <code>base</code> and <code>text</code> rendering, Red's codebase received a <a href="https://github.com/red/red/pull/3683">contribution</a> for testing of image-related features on Windows platforms, accompanied by more than 300 tests. Coupled with our <a href="https://github.com/red/red/tree/master/modules/view/backends/test">"null" testing backend</a> (covered in some detail in <a href="https://www.red-lang.org/2017/07/063-macos-gui-backend.html">here</a>) this casts a wide net for catching regressions in View.<br />
<br />
After the <a href="https://www.red-lang.org/2018/12/064-simple-gc-and-pure-red-gui-console.html">0.6.4 release announcement</a>, many users checked out rich-text support and gave Red's <a href="https://github.com/red/red/wiki/%5BDOC%5D-Rich-Text-support">Rich-text dialect</a> a proper spin, with numerous <a href="https://github.com/red/red/pull/3698">bug fixes and tweaks</a> being submitted in the process.<br />
<br />
<br />
<h3>
</h3>
<h3>
</h3>
<h3>
</h3>
<h3>
Editing, with a capital E</h3>
<br />
As you can see from the above, native GUI support is one of our strongest aspects, which piques the interest of many community members.<br />
<br />
Another special feature that catches everyone's fancy is Red's <a href="https://en.wikipedia.org/wiki/Homoiconicity">"code as data"</a> nature. But, while giving Red enormous expressive power (metaprogramming and the creation of domain-specific languages, DSLs for short), it makes some things trickier — with the boundary between code and data being blurred, and multiple dialects (our term for embedded DSLs) at one's disposal, it is extremely challenging to assign any particular meaning to a given piece of code (data!); both for humans (esp. newr Red programmers) and automatic tools, such as syntax highlighters, static code analyzers, debuggers and linters.<br />
<br />
Historically, this was the main argument when it came to Rebol's tooling. Sprinkling your code with <code>print</code> and <code>probe</code> is still the most common debugging method, and error reporting boils down to stack traces rather than source text info; text editing support can accommodate only cosmetic niceties like "keywords" highlighting and indentation, covering only general subset of the language.<br />
<br />
Being a direct descendant of Rebol, we carried over this bag of concerns, aiming to resolve them at some point in development. A quite natural answer to points raised would be an editor written in pure Red and its dialects, integrated in our self-contained toolchain.<br />
<br />
And while writing a basic version of such editor is <a href="https://www.red-lang.org/2016/07/eve-style-clock-demo-in-red-livecoded.html">rather trivial</a>, such a naive approach leaves much to be desired, and keeps questions about proper tooling surrounded with an air of uncertainty, begging for a detailed investigation. Thankfully, this is where community projects and experiments come to the rescue!<br />
<ul>
<li>
<strong><code><a href="https://github.com/rebolek">@rebolek</a></code></strong>'s <a href="https://gitlab.com/rebolek/values">Values</a>, announced at last year's <a href="https://www.red-lang.org/2018/11/redcon-1-and-ethereum-devcon-4-world.html">RedCon</a>, is a sophisticated <a href="https://en.wikipedia.org/wiki/Structure_editor">projectional editor</a> with Vim's flavor and an ingenious schtick — rather than working with textual form, it <code>load</code>s a program and directly manipulates its internal data structure (a <code>block!</code> of <em>values),</em> reflecting all changes back in the text area.<br /><br />Idiomatically written, and offering sheer power in a few keystrokes, it struck us in awe back at RedCon, and stimulated a lot of technical discussions. With our <a href="https://www.red-lang.org/2018/12/064-simple-gc-and-pure-red-gui-console.html">new GUI console</a>'s plug-in API on its way for 0.6.5 release, we are eager to see how it will get integrated in day-to-day user experience, and what fresh insights it can bring to all of us.<br />
</li>
<li><strong><code><a href="https://github.com/toomasv">@toomasv</a></code></strong> is well-known in our community as an avid Draw and View tinkerer, and his recent <a href="https://github.com/toomasv/syntax-highlighter">experiment</a> only firms this reputation.<br /><br />Datatype-aware syntax highlighting, contextual help, auto-completion, step-by-step evaluation and bunch of other niceties — all in ~1K lines of code! Magic, you say? No, Reducing at its finest. ;-)</li>
</ul>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://vooglaid.ee/red/syntax4.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://vooglaid.ee/red/syntax4.gif" data-original-height="497" data-original-width="704" height="449" width="640" /></a></div>
<br />
<br />
Projects like this (and all others that we didn't get a chance to mention) aren't just eye-candy for the public — they help us, core developers, to evaluate all options and to revisit old ideas in a new light. Besides, it's extremely rewarding to see Red (in its <em>alpha</em> stage) being put to real use.<br />
<br />
Once again, our cheers go to all the people who dedicate their time and efforts to support us. No matter how small your contribution, it makes Red stronger and readier for the prime-time: day by day, bit by bit.<br />
<br />
<h2>
</h2>
<h2>
</h2>
<h2>
C3 project</h2>
<br />
Since the core team relocation is our ongoing No.1 priority, development of the C3 project got sidetracked a bit. There are still a few final touches that need to be done (e.g. finishing AST parser, <code>hex!</code> and <code>bigint!</code> datatypes support) but, aside from that, the first alpha version of compiler is almost ready for a public release. The first 90% of a task takes 90% of the time, the last 10% takes the other 90%.<br />
<br />
In the meantime, we have one appetizer to serve: RedCon talk about C3! Due to technical issues and time constraints, it became available only recently, and was captured on a nearby web-camera. Because of that, video quality and recording angle are so-so, but the good news is that you can gain a deeper insight into the blockchain aspect of Red... and witness the talking belly. :-)<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/HwYkrHDe0M8/0.jpg" frameborder="0" height="266" src="https://www.youtube.com/embed/HwYkrHDe0M8?feature=player_embedded" width="320"></iframe></div>
<br />
<br />
As evident from the talk<a href="https://www.red-lang.org/2019/02/january-2019-update.html#footnote-2"><sup>2</sup></a>, C3's design proved to be much more challenging than we initially anticipated, muddled by pesky EVM (Ethereum Virtual Machine) constraints (like gas), lack of good reference documentation, complex tooling and blockchain hype haze.<br />
<br />
At this point, we're convinced that <em>at least two</em> layers are required for C3 to be a sound smart-contract development solution:<br />
<ol>
<li>The low-level one, named <em>C3/System</em>, is a language with simple semantics (close to Solidity, but without foot-shooting), serving as a base for building high-level dialects on top of it. We also want to implement a couple of experimental features, like off-chain Cron-like scheduler of Smart Contract execution and embedded FSM (finite-state machine) sub-dialect.</li>
<li>Possibly <em>multiple</em> high-level layers, each covering different blockchain aspect and tailored to specific solutions: creation of crypto tokens, gambling, in-game collectibles, legal contracts and business logic, bids and auctions, asset tracking... who-knows-what people may come up with.</li>
</ol>
Despite all the media craze surrounding blockchains, the technology itself is still in a nascent state, constantly searching for a real-world use-case. Because of that, navigating this new and constantly shifting space (not to mention building a development platform like C3) is like sailing in stormy weather.<br />
<br />
In light of the above, we owe much to<a href="https://github.com/9214"> <strong><code>@9214</code></strong></a> for his help with C3 preliminaries and conducting in-depth research on Smart Contract languages. His constant background presence affirmed many of our assumptions, and guided us past many reefs and pitfalls.<br />
<br />
Back in the day, our <a href="https://www.red-lang.org/2017/12/leaping-into-future-red-goes-blockchain.html">"Red goes blockchain"</a> announcement generated much controversy, and prompted many people (both community members and outsiders) to cast their doubts on this decision. Some go as far as to dismiss the Red project entirely, based on our new roadmap.<br />
<br />
Yet, very few realize that, in some sense, we're picking up the game that Rebol left — with Carl's conception of <a href="http://www.rebol.com/news/news3404.html">X-Internet</a><a href="https://www.red-lang.org/2019/02/january-2019-update.html#footnote-3"><sup>3</sup></a> replaced by decentralized consensus-driven P2P networks, and EVM instead of <a href="http://www.rebol.com/ios/ios-intro.html">IOS</a>. Indeed, Smart Contracts are just very limited...slow <a href="http://www.rebol.com/reblets.html">Reblets/services</a> in disguise ;-) And behind blockchain fatigue lies the brave new world, where Red can be the missing puzzle piece and stepping stone towards simple, human-friendly tooling.<br />
<br />
All in all, we hold to our beliefs, and going to play this card, aiming to hit the jackpot. But we're also not putting all our eggs in one basket. Bear with us as we tread forward, and stay tuned for future updates!<br />
<br />
<h2>
</h2>
<h2>
</h2>
<h2>
Wrapping-up</h2>
<br />
January, with its holidays and family reunions, is nothing but a calendar sheet. The work on Red never ceases, spans through weekends, occupies all time and space (both physical and mental) and now reaches across the globe. For some, it became the animating principle of their lives.<br />
<br />
With our hard work at stake, and with 1.0 release in near sight, encounters with the outside world get more frequent and progressively challenging — criticism and cross-checks with reality are important, but it's equally important to apply a fair yardstick, and judge the project on its own merits.<br />
<br />
That's why, when it comes to media coverage and public discussions, we expect outsiders to stay open-minded and not to cast aspersions lightly. We want Red to succeed, and welcome technical debate and comparisons with other projects (like, e.g., <a href="https://www.reddit.com/r/redlang/comments/aebxct/contrast_red_with_racket/edoerll/">Red vs. Racket</a>; again, kudos to <strong><code>@9214</code></strong> for writing such a well-articulated response). But, more often than not, discussion threads turn into exercises at damage control, with constructive points being hamstrung and torn to pieces in a cross-fire.<br />
<br />
Our mission to fight software complexity is challenging, and not for the faint-hearted. Remember — the best way to prove one's case is not in word, but in deed... and the strongest argument is a tech-savvy pull-request. ;-)<br />
<br />
Come to think of it, I can't help remembering Pierre Rabhi's tale:<br />
<blockquote class="tr_bq">
Once upon a time, the scorching fire swallowed the forest. All the animals, terrified and powerless, watched it destroying their homes from afar. Only little Hummingbird kept flying back and worth between the fire and nearby pond, carrying tiny drops of water in its beak and throwing them in the flames. </blockquote>
<blockquote class="tr_bq">
Irritated by bird's constant flickering, grouchy Armadillo yelled: "Hummingbird! Don't you see that your pathetic attempts won't save the forest?"</blockquote>
<blockquote class="tr_bq">
"I know.", answered Hummingbird, "but I'm doing my part."</blockquote>
<br />
Hectic 2018 barely passed by, but we're already steering wheels on our <a href="https://www.red-lang.org/2019/01/full-steam-ahead.html">new roadmap</a>. There is more news to announce, once the dust has settled. We'll be in touch. Until then, Happy Reducing!<br />
<br />
<em>-- Post by 9214, edits and amendments by Gregg</em><br />
<h2>
<em><br /></em></h2>
<h2>
<em>Footnotes</em></h2>
<ol>
<li><a href="https://www.blogger.com/null" name="footnote-1"></a> Remember that you can track progress of Red projects (both community-driven and official ones) at <a href="https://progress.red-lang.org/"></a><a href="https://progress.red-lang.org/">https://progress.red-lang.org/</a></li>
<li><a href="https://www.blogger.com/null" name="footnote-2"></a> Reminder: C3 presentation slides in PDF format are available <a href="https://static.red-lang.org/C3/Red-C3.pdf">here</a>.</li>
<li><a href="https://www.blogger.com/null" name="footnote-3"></a> "X" here stands for eXecutable and eXtendable.</li>
</ol>
<br />Unknownnoreply@blogger.com3tag:blogger.com,1999:blog-5936111837781935054.post-85281191213453402012019-01-06T11:41:00.000+01:002019-01-06T11:41:30.905+01:00Full steam ahead!<span style="font-size: large;">The past year</span><br />
<br />
It has been a tough year 2018, filled with roller-coasters and instability. The constant dropping of the crypto markets has forced us to change our plans pretty much every three months, in order to cope with that market uncertainty and our diminishing means. Though we have taken measures to ensure the proper funding for Red project for the next few years, so no worries on that side. All that has slowed down work on Red's core and even our new C3 language. That said we still have managed to make some great progress in the past year, e.g.:<br />
<br />
<ul>
<li><a href="https://www.red-lang.org/2018/10/red-wallet-alpha-2.html" target="_blank">RED Wallet</a> released: a good showcase of what can be achieved in a small <a href="https://github.com/red/wallet" target="_blank">codebase</a> using a truly full-stack programming language like Red, from USB and hardware keys drivers to blockchain network interfacing and cross-platform UI!</li>
</ul>
<ul>
<li>Garbage Collector: a simple mark & sweep, compacting GC for Red! That is a big step towards a completed production-ready Red version 1.0.</li>
</ul>
<ul>
<li>Red console fully re-implemented in Red itself, allowing <a href="https://www.red-lang.org/2018/12/064-simple-gc-and-pure-red-gui-console.html" target="_blank">new features</a> like syntactic coloring and user plugins.</li>
</ul>
<ul>
<li>New rich-text display widget and related <a href="https://github.com/red/red/wiki/[DOC]-Rich-Text-support" target="_blank">RTD dialect</a>.</li>
</ul>
<ul>
<li>DPI-independent support for Red/View.</li>
</ul>
<ul>
<li>Preliminary port! and bigint! datatypes implementations.</li>
</ul>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">eth://</span> port for interacting with the Ethereum blockchain (to be released publicly very soon).</li>
</ul>
<ul>
<li>PoC async <a href="https://github.com/qtxie/red/tree/ev/tests/IO" target="_blank">tcp/udp implementation</a> with client and server-side support.</li>
</ul>
<ul>
<li>350 tickets closed, among which 136 are bugfixes.</li>
</ul>
<ul>
<li>RedCon1 held in Prague in November!</li>
</ul>
<ul>
<li>Extended Red team with now about 12 people in total.</li>
</ul>
<br />
We are also working on structuring the Red Foundation and have a new recruit working on tokenomics. So the tokens retro-distribution to kick-start tokenomics should happen very soon. <br />
<br />
<span style="font-size: large;">The plan for 2019</span><br />
<br />
We are starting the year with big targets as we need some last big pieces of the Red project to be implemented to reach both our technical and business goals. Here are the main tasks we will handle during this new year:<br />
<br />
<ul>
<li><b>C3 toolchain release</b>: the C3/System language spec was defined last October and work on the compiler and related toolchain is on-going. This is our main short-term priority as we want to test our business options in the smart contract market while it is still in its early days.</li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Red full async I/O support</b> (0.7.0 milestone): we already have a PoC for async tcp/udp support, with Rebol-like event handling API. There is still a need for more design work (on-going right now) in order to provide the best possible API for Red users. For now, we are trying to avoid going full-force into the deep territory of defining a general concurrency support (it's scheduled for 0.9.0 milestone). I'm not yet sure if we can escape stepping into it anyway if we want to go beyond the Rebol async model.</li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Full Android support for Red</b>: we have a <a href="https://www.red-lang.org/2018/04/sneak-peek-at-red-on-android.html" target="_blank">preliminary implementation</a> that has proven to be very promising! With the integration of async I/O and ports, we will be able to complete the wrapping of the Android API using decent human-friendly abstractions. We need to advance Android support to at least beta level by the end of this year. This is one of the major goals we need to achieve.</li>
</ul>
<div>
<br /></div>
<ul>
<li><b>Red/Pro</b>: it is coming this year! It will be our first commercial product and set of online services targetting both individual developers and enterprises. It will include a new backend for Red's toolchain, providing a state-of-the-art optimizing layer and support for many new platforms, including 64-bit ones (though full 64-bit Red support will probably be for next year). That new backend will provide an alternative to our current code emitter and linker, the rest of the toolchain remaining the same. The current Red toolchain will be known as the "community" version, and continue being the main development branch.</li>
</ul>
<div>
<br /></div>
<div>
In addition to all that, a 0.6.5 milestone for Red is in the plans, aiming at making the Red console the main focus for Red users by providing:</div>
<div>
<ul>
<li>prebuilt versions of the Red console.</li>
<li>ability to invoke the Red toolchain from the Red console.</li>
<li>a plugin API to extend the console in many ways.</li>
</ul>
</div>
<br />
In order to better focus on these tasks and better interact with our new team members in Europe, we will be relocating our core team from different parts of the world to Europe (Red Foundation is located in Paris). We also plan to make a RedCon2 developer conference in Europe later this year.<br />
<br />
Many thanks to all the people who have contributed or helped in any way with advancing Red and managing the community. How do we climb the tallest mountain? One step at a time! And the top is in sight now, we're almost there. ;-)<br />
<br />
Happy New Year to all, and keep up the great work you are all doing!<br />
<br />Nenad Rakocevichttp://www.blogger.com/profile/06600325626233743055noreply@blogger.com7tag:blogger.com,1999:blog-5936111837781935054.post-69490684112884617882018-12-07T19:12:00.001+01:002018-12-30T00:13:21.016+01:000.6.4 Simple GC and Pure Red GUI ConsoleIt's been a long time coming.<br />
<br />
While work on Red has progressed every day, work on the blockchain aspects of the project, and their associated logistics, came at a time when that workload temporarily prevented us from making new, formal/stable releases. There was a key feature that needed to be in place, and it wasn't a small one. Thanks for bearing with us!<br />
<br />
Fixes and features kept rolling out on a regular basis, so most active community members were not affected by the lack of a fresh new "official" release. It did, however, become an issue for people new to Red, because as the stable download for 0.6.3 became more and more out of date, users would report bugs that had already been fixed! So now 0.6.4 should see us on the same page.<br />
<br />
We'd like to promise we'll never go that long again between official releases, and that is our goal, but we hope you'll roll with the punches, as we do the same. The best way to keep up to date is to use the automated builds, so you're always current. Some people will want to stay with the stable version for various reasons, and that's fine too.<br />
<br />
This new Red release contains 1'116 commits, closes <a href="https://github.com/red/red/issues?utf8=%E2%9C%93&q=is%3Aclosed+milestone%3A0.6.4">167 issues</a> tagged for this milestone, and adds a lot of small features. Looking at the list of bugs fixed, and features added, is a great reminder of all Red has to offer.<br />
<br />
<br />
<h2>
1. Simple GC</h2>
<br />
The main feature for 0.6.4 is what we call the Simple GC (Garbage Collector). A more advanced GC is planned for the future, but this release will make it easier to create long running processes, without having to be aware of all your allocations, or manually reusing series to conserve memory. Things just got a whole lot easier!<br />
<br />
There are 2 functions related to this, that you can use:<br />
<br />
1. You can use <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">stats</span> to see the amount of memory in use, and its <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/info</span> refinement provides more details to aid deeper debugging if needed.<br />
<br />
2. The second function is called <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">recycle</span>, which triggers a manual garbage collection pass, rather than waiting for internal heuristics and timing events to do so. <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">Recycle</span> has 2 refinements: <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/off</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/on</span>. They allow you to disable the garbage collector, and re-enable it. This gives you control in scenarios where you want to manage things more directly yourself.<br />
<br />
It's also important to note that the GC operates at the level of Red values, and <i>does not</i> affect Red/System code. Red/System is a C level language, where you are responsible for allocating and freeing memory, and where you may need precise manual control over those actions. (It's possible to add an optional GC for Red/System in the future, but it's not currently on our roadmap.)<br />
<br />
<br />
<h2>
2. Simple Isn't Always Easy</h2>
<br />
Our main goal is to fight software complexity, but nothing can eliminate it completely.<br />
<br />
Our small size belies the number of details that have to be considered, interactions between the large number of datatypes, and how to make each most useful with the standard library of functions. And then there's the reactive GUI system, with its <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">draw</span> dialect, both very flexible and powerful, which leads to more combinations that need to be handled, and edge cases to uncover. We accept the effort needed, because we are our own toolchain. A great deal of work went into the new GUI console, with a lot more planned.<br />
<br />
Apple's talk of eliminating all 32-bit support could impact us. We are currently 32-bit only, and need to decide the best path for adding 64-bit support, should we elect to do so. libRed is also kept up-to-date in this release as well, for integration from other tools and languages. Additional improvements were made in the ELF format emitter.<br />
<br />
We still face being flagged by a few big AV products, and these false positives hurt our image. It's frustrating to be marked clean one day, then have their heuristics change and flag you the next. <i><u>Keep reporting the problem to them</u></i>, and check VirusTotal to see how they compare. It's just one more challenge, because we're small, and use our own toolchain, so they don't know what to do with us.<br />
<br />
Not all changes are big, and a lot of work also goes into documentation and tests on a regular basis. The code is only part of the project. A lot of our time this past year has been spent on logistics related to business, and making the Red project sustainable long term.<br />
<br />
<br />
<h3>
2.1: Operating System Support</h3>
<br />
Providing the maximum number of features across platforms is not easy, and we will need to freeze support for older OSs, like Windows XP, so we can focus efforts on the systems used by more people today.<br />
<br />
Keeping up with mainline changes on all the operating systems we support is a challenge itself. We've kept ARM soft float working, and FreeBSD support has remained largely intact. We got a report about it breaking recently, and a fix went out the next day. FreeBSD isn't critical for us, but that fast response shows how a smaller and simpler code base can help.<br />
<br />
<br />
<h3>
2.2: A REPL In Time</h3>
<br />
Having an instant-on REPL is enormously helpful to newcomers and experts alike, and we'll build on this new foundation to add more features.<br />
<br />
One change we expect to make for 0.6.5 is to change the download binary, so REPL use is even easier. Right now, when you download Red, you get the toolchain. The first time you run it, it builds the REPL console on your system locally. That takes some time, and isn't a great first user experience. So as we move forward, we're considering making the REPL the download, and then when you go to compile, if Red sees you haven't installed the toolchain, it will download it at that time. We'll keep you updated as we try our alternatives.<br />
<br />
<br />
<h2>
3. Rich Text</h2>
<div>
<br />
There's a new style in fashion this season: <span style="font-family: "courier new" , "courier" , monospace;"><span style="color: #6aa84f;">rich-text</span>.</span> </div>
<div>
<br /></div>
<div>
Full documentation is forthcoming, but you can see the source code for the RTD (Rich Text Dialect) in <a href="https://github.com/red/red/blob/master/modules/view/RTD.red">%red/modules/view/RTD.red</a> (only ~150 lines), and reference information in the <a href="https://github.com/red/red/wiki/[DOC]-Rich-Text-support" target="_blank">wiki.</a><br />
<br />
Eight attributes are currently supported. More can be added, but these gives you a great deal of control, and are by far the most commonly used: <span style="font-family: "courier new" , "courier" , monospace;">bold, italic, underline, strikethrough, text color, background color, font-name, and font-size. Border</span><span style="font-family: inherit;"> is planned but not yet implemented, as are some advanced options for the above attributes (e.g. wavy underline as is often used for spell checking markup). Attributes can be nested, of course.</span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;">@toomasv put together a small editor, which is about 400 lines of code, but since the rich-text style doesn't have built in keyboard handling yet, he had to roll his own, which is the majority of the code. </span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div>
<span style="font-family: inherit;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJhe6Wv3xPOWCrF4a7XpCEfKP-HScEVPB8ZiuCgudBnAiijVtKgdjD49pmG5D5iJOJVKA_Cj-mZSyWO86vQGrh7KYU_jF2zMGLOuT2Wha6ed05g5dtoQBihHCf438WuRi4FaDf9BN50Ck/s1600/RTD-demo-1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="313" data-original-width="352" height="355" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiJhe6Wv3xPOWCrF4a7XpCEfKP-HScEVPB8ZiuCgudBnAiijVtKgdjD49pmG5D5iJOJVKA_Cj-mZSyWO86vQGrh7KYU_jF2zMGLOuT2Wha6ed05g5dtoQBihHCf438WuRi4FaDf9BN50Ck/s400/RTD-demo-1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both;">
Using the <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> style directly in code is easy too. The <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">data</span> facet just needs to contain RTD commands and, internally <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rtd-layout</span> is called to render those commands into the <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">text</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">data</span> facets of the face. You can also create a <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> face, and later call <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rtd-layout</span><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/with</span> to update commands and render them.</div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
While Red prefers plain words for markup elements, RTD also supports HTML style tags, for convenience. (Note, RTD is <i>not</i> designed to be HTML compatible, and it's not a goal on our roadmap.) </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
You can see different approaches to formatting, below: the first three examples all produce the same output, and show how attributes can be applied to blocks of commands, which is more idiomatic in Red. That is, when you have a formatting command followed by a block, everything in that block has the formatting command applied. It makes viewing the formatting structure much easier, and also eliminates attribute ordering issues. </div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
The last example below shows how a group of attributes can be expressed as a single <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">path!</span> value, applying all of them to the block that follows.</div>
<div class="separator" style="clear: both;">
<br /></div>
<pre class="code"> view [
rich-text data [
<i> <b> "Hello" </b> <font> 24 red " Red " </font> blue "World!" </i>
]
rich-text data [i b "Hello" /b font 24 red " Red " /font blue "World!" /i]
rich-text data [i [b ["Hello"] red font 24 [" Red "] blue "World!"]]
rich-text data [i/b/u/red ["Hello" font 32 " Red " /font blue "World!"]]
]</pre>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSMJKcoVGm6uQ201TfZGJ0txqGWjk8b5C9-vIyAruL0zjpzpy2i8L6tPnPkXuHSZqP8xSVR9il160nER7-zWrg-LNY9Mfax3rQX8JdeiStwcsPA44hEbo7oBJRG8LyypGmAJqQVDuM2pc/s1600/RTD-demo-2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="203" data-original-width="656" height="122" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSMJKcoVGm6uQ201TfZGJ0txqGWjk8b5C9-vIyAruL0zjpzpy2i8L6tPnPkXuHSZqP8xSVR9il160nER7-zWrg-LNY9Mfax3rQX8JdeiStwcsPA44hEbo7oBJRG8LyypGmAJqQVDuM2pc/s400/RTD-demo-2.png" width="400" /></a></div>
<div class="separator" style="clear: both;">
<br /></div>
<div class="separator" style="clear: both;">
This is only the most basic use. See @toomasv's <a href="https://gist.github.com/toomasv/8fcd1bbd7de9451e7789dbdf5ae35881">example gist</a> for more uses, and consult the wiki doc to see how to query information like line height and caret position. Beyond that <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> is already hardware optimized and supports multiple rendering areas in a single face, which makes it suitable for quite complex layouts of richly formatted text. You can also combine <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">draw</span> commands in a <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> face, for even more power.</div>
<div>
<span style="font-family: inherit;"><br /></span>
<span style="font-family: inherit;"><br /></span></div>
<h2>
4. GUI Console: More Than Meets the Eye</h2>
<div>
<br /></div>
<div>
While the GC took the spotlight in this release, the console also saw huge changes and improvements. The biggest change isn't visible to the naked eye: the new GUI console is now implemented in pure Red, using the <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> face style. This sets the foundation for building more complete Red editors, tools, and IDEs. Basic features in place right now:<br />
<br /></div>
<div>
<ul>
<li>History and Undo/Redo</li>
<li>Copy/Paste support</li>
<li>Hot-keys and shortcuts</li>
<li>Syntax Coloring</li>
<li>Auto-completion</li>
</ul>
<div>
<h4>
</h4>
<h4>
</h4>
<div>
<br /></div>
<h3>
4.1: Console Hot Keys</h3>
</div>
</div>
<div>
<br /></div>
<div>
<div>
<ul>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + Backspace delete previous word</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + Del delete next word</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + Up/Down scroll up/down</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + C copy</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + V paste</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Shift + Insert paste</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + X cut</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + Z undo</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + Y redo</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + L clear screen</span></li>
<li><span style="font-family: "courier new" , "courier" , monospace;">Ctrl + K delete the whole line</span></li>
</ul>
</div>
</div>
<div>
<h4>
</h4>
<h4>
</h4>
<div>
<br /></div>
<h3>
4.2: Syntax Coloring</h3>
</div>
<div>
<br /></div>
<div>
You can turn syntax highlighting on with the following command:</div>
<div>
<br /></div>
<div>
<span style="background-color: rgba(192 , 201 , 200 , 0.2); font-family: monospace; font-size: 14px;"> gui-console-ctx/terminal/color?: yes</span></div>
<div>
<br /></div>
It is not dynamic, so you may find the colors don't work well with the colors you've selected in the Settings dialog for the console. There is no support for customizing themes yet. If you want to experiment with changing colors, you have to manually set values in <span style="background-color: rgba(192 , 201 , 200 , 0.2); font-family: monospace; font-size: 14px;">gui-console-ctx/terminal/theme</span>.<br />
<h4>
</h4>
<h4>
</h4>
<div>
<br /></div>
<h3>
4.3: Auto-Completion</h3>
<div>
<br />
This feature can support completion for 3 types of values: words, files, and paths. </div>
<div>
<br /></div>
<div>
For words, if you type a word and hit the Tab key, one of two things can happen. Either there's a single match for what you've typed so far, which will then be auto-completed for you, or all matches will be shown.<br />
<br />
Try typing <b>p <tab></b>. You'll get a list of all words in the system that start with the letter P.<br />
<br />
Now, type <b>pa <tab></b>. The <span style="font-family: "courier new" , "courier" , monospace;">help</span> function works in a similar way, but with different output.</div>
<div>
<br /></div>
<div>
Filename completion works the same way. If you type a file value, starting with <span style="font-family: "courier new" , "courier" , monospace;">%</span>, the letters you type are then prefix matched against the current directory in the file system. It works very much like a system shell's autocomplete feature, but built directly into the Red console. Snazzy!</div>
<div>
<br /></div>
<div>
Path completion is a little different:<br />
If you type a path, e.g., <span style="font-family: "courier new" , "courier" , monospace;">system/</span>, and then hit the Tab key, the console will display the list of words available for that path value.<br />
If the value is an object, the words of that object will be displayed.<br />
If it's a function, the possible refinements will be listed.<br />
<br />
For objects, it makes it easy to drill down into nested structures. For functions, it offers basic help, but doesn't do anything advanced yet. For example, if you use more than one refinement, auto-completion will still list every refinement available, rather than removing those you've already used from the list.<br />
<br />
<br /></div>
<div>
<h2>
5. Other Highlights & Changes in 0.6.4</h2>
<br />
Here are just a few more details:<br />
<br />
<h3>
5.1: View and VID</h3>
</div>
<div>
<ul>
<li>DPI support added, which is great, but also raises new OS system scaling questions, currently being reviewed</li>
<li><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">rich-text</span> style added</li>
<li>Password mode added to <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">field</span> style (not supported in VID yet); include <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">'password</span> in face flags</li>
<li><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">Make-face</span> helper function added</li>
<li>RTD (Rich Text Dialect) support (MacOS & Windows)</li>
<li>Ctrl+A hot-key works in <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">area</span> now</li>
<li>Text selection support improved for <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">area</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">field</span></li>
</ul>
<div>
<br /></div>
</div>
<div>
<h3>
5.2: Core Language</h3>
</div>
<div>
<ul>
<li>Added <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/deep</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/case</span> refinements to <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">replace</span></li>
<li>Simple I/O now supports all HTTP verbs</li>
<li>Added <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">decompress</span> native, with support for <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">gzip, zlib</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">deflate</span> formats</li>
<li>Adler32 support added to<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: #6aa84f;">checksum</span></span></li>
<li>Maps now support<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: #6aa84f;">binary!</span> </span>values as keys</li>
<li>Latin-1 strings can now be decoded</li>
<li><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">Trim</span> now supports <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">binary!</span> values. By default, only <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">null</span> values are removed. If you use the <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">/all</span> refinement, whitespace chars are trimmed as if it were a string</li>
<li>Basic <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">query</span> action added for files, which returns their timestamp</li>
<li>Added <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">offset-to-char</span> function. Given a face and a coordinate, it returns the corresponding character position</li>
<li>Added <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">os-info</span> func. It returns detailed operating system version information, which can be helpful when debugging and submitting tickets (used by <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">about/debug</span>)</li>
<li><i>Very</i> simple <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">alert</span> function added; more are available as community experiments</li>
<li><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">Enbase/Debase</span> now support base 58</li>
<li><span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">Sum</span> and <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">average</span> functions added</li>
</ul>
</div>
<div>
<br />
<br />
<h2>
6. Looking Ahead</h2>
<div>
<br /></div>
<div>
<ul>
<li>Full I/O is our next major milestone, and some other key features, like async I/O and a networking port model, will come with that.</li>
<li>Android: We've changed up the roadmap a bit, so Android will come after Full I/O, because it will benefit from having it, and save rework if done the other way around. This is a very important -- dare we say, critical? -- milestone, and one we will put all our efforts behind in the coming months.</li>
<li>Blockchain Aspects: The RED Wallet is expected to get some new features, and possibly a new look. We need a dedicated product manager for it, and another team member who can focus on it. The C3 project is going full steam, and we'll be organizing resources to keep that moving as we redirect key players to Red's core early next year.</li>
<li>We have an experimental <span style="color: #6aa84f; font-family: "courier new" , "courier" , monospace;">money!</span> datatype, based on Douglas Crockford's <a href="http://www.dec64.com/">Dec64</a> design. It will have a limited value range initially, due to our 32-bit foundation, but it's a first test, and can be used to see how it works in action. We believe we're the first language to implement it.</li>
<li>JSON and CSV codecs. These are works in progress, and their design had led to discussion of how best to do certain things outside them -- for example, how to represent JSON<span style="font-family: "courier new" , "courier" , monospace;"> <span style="color: #6aa84f;">null</span></span> values in Red maps, which isn't yet possible.</li>
</ul>
<div>
<br /></div>
</div>
</div>
<div>
We're enormously proud of how Red's growth and community support have accelerated. For a project like ours -- ambitious, idealistic, open-source -- to have garnered interest in the way we have, has been greatly rewarding. We look forward to a 2019 rich with REDucing software complexity, yet celebrating the increasing power and impact of the language and the movement.<br />
<br /></div>
Unknownnoreply@blogger.com6tag:blogger.com,1999:blog-5936111837781935054.post-33732005625495755902018-11-22T20:23:00.000+01:002019-01-02T10:05:32.953+01:00RedCon 1 and Ethereum DevCon 4 World Report<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL2kAdLwJGsqMMI5Je0n5ESZNWz5nF7uocv-YuCssv8LWQRDkdHpXPCQAwcau2hKd_DXnsD4E7FvbBbvE4a6yYurRlIcpp6FwqpOVSjlSLmKW3voNvzHqb96kSPKEqSFgGK7t7r7IfOTc/s1600/devcon-entrance.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="612" data-original-width="816" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL2kAdLwJGsqMMI5Je0n5ESZNWz5nF7uocv-YuCssv8LWQRDkdHpXPCQAwcau2hKd_DXnsD4E7FvbBbvE4a6yYurRlIcpp6FwqpOVSjlSLmKW3voNvzHqb96kSPKEqSFgGK7t7r7IfOTc/s640/devcon-entrance.jpg" width="640" /></a></div>
<br />
Happy November, Red community! We’ve been very busy. Beginning in September, Nenad and Qingtian joined Gregg in Idaho for 6 weeks, in preparation for Ethereum DevCon 4, and to set some things in motion for a possible U.S. Red presence. Challenges continued to appear at every turn, which kept us on our toes. By the end of October, we were ready to hit Prague for the DevCon, and to have our first Mini RedCon with some of the team. Here’s what happened.<br />
<br /></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<h3>
Ethereum DevCon 4</h3>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirjULozO4AZIKUTc7V3Rn7566VG6tacpOJoDaxpTD0g_7fcOYpX91-FZk_rxrWTg4Zb7w5Zk-ukTV6XUIdeBx4oY40XPYljY1tRK6ryvsla4rTNAaRWt2SCh9i1XvBh1ZdXA4_D-q69lE/s1600/NRGI-devcon.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="612" data-original-width="816" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirjULozO4AZIKUTc7V3Rn7566VG6tacpOJoDaxpTD0g_7fcOYpX91-FZk_rxrWTg4Zb7w5Zk-ukTV6XUIdeBx4oY40XPYljY1tRK6ryvsla4rTNAaRWt2SCh9i1XvBh1ZdXA4_D-q69lE/s640/NRGI-devcon.jpg" width="640" /></a></div>
<div>
<br /></div>
</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
It was a lot of work, leading up to Ethereum DevCon4, and we tried every avenue to sponsor and support the Ethereum community, including submission of possible presentations in every category. Unfortunately, we were not selected to present, as there were only about 50 presentation slots, and over 1000 submissions. Still, we did the best we could, and attended as many sessions as possible that we thought were valuable.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
Some of those sessions were very good, some less so. The quality varied widely, as did the type of attendee. It was surprising to find that only a tiny percentage were actual developers, which is our target audience, considering this was advertised as a "Dev" Con, and 70% of the tickets were reserved for "BUIDLers". <a href="https://www.red-lang.org/2018/11/redcon-1-and-ethereum-devcon-4-world.html#footnote-1"><sup>1</sup></a></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
We noticed a systemic issue at DevCon: There was a lot of hype surrounding many projects, but when it came to actual follow-through, it was hard to find programmers at the DevCon. The hardcore deep-devs on the EVM and language development side were the biggest "clusters" we found. Those writing smart contracts and building actual working project infrastructure were few and far between.<a href="https://www.red-lang.org/2018/11/redcon-1-and-ethereum-devcon-4-world.html#footnote-2"><sup>2</sup></a> Some commentary on that in our footnotes.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
Nevertheless, we did find some key builders (real ones!) and creators who had valuable insights:</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<ul>
<li>One of the people who takes this seriously, and seems to be carefully walking the line of professionalism and community culture, is Sid Coelho-Prabhu of CoinBase. His talk was professional, on-point, clearly stated some of the same problems we see, but also played well to the crypto- and Ethereum-culture crowd. He has a positive approach and is realistic, which was great to see. (<b>Best in Show)</b></li>
<li>The DevEx Breakout EVM Panel consisted of a good cross section of EVM developers, some core, some higher-up, formalists, and others. It gave us a lot of insight into the history of the EVM, challenges it faces, and their thoughts on future ideas. </li>
<li>Another highlight was our meeting with the Solidity and Vyper teams. While we have different goals, let it be known that we have the highest respect for their skills, and the kind of people they are. As outsiders, potentially competitors, they welcomed us, shared insights, and both challenged and supported ideas we shared about our goals and design elements of Red/C3. Our deepest thanks to Alex, Christian, Jacque, and Bryant for seeing us as walking the same road, sharing the goal of making it possible for people to use and experiment with Ethereum on the Distributed Executable Code (Smart Contract) side, which is separate from the EVM team(s). </li>
</ul>
</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU2ekvPiw2J2QSmVny8ENv_HjEMf1A22SfwvfTcbES9okyHHHQQiQnY6gMHKAHUt9t41M2HX384sP0Ro-hRVqQ1Vqk5Txrg59Cjruwth2iNz4awzyZQYHs1-I-nzgV9i0UiKIh1jJ5fqs/s1600/lang+team+meetup.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="612" data-original-width="816" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhU2ekvPiw2J2QSmVny8ENv_HjEMf1A22SfwvfTcbES9okyHHHQQiQnY6gMHKAHUt9t41M2HX384sP0Ro-hRVqQ1Vqk5Txrg59Cjruwth2iNz4awzyZQYHs1-I-nzgV9i0UiKIh1jJ5fqs/s640/lang+team+meetup.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
Secret Language Design Meeting</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
On a related note to DevCon, Prague is a beautiful city, and the Ethereum Foundation chose well in selecting it as the host. The Congress Center staff were efficient and professional, and the overall logistics were very well managed. A thank you to the organizers for all their hard work.<br />
<br /></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<h3>
RedCon 1</h3>
</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
The day after DevCon ended, we held our first RedCon in Prague, bringing together some of our core team, contractors, and new people who were interested in Red and Red/C3. </div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
We planned an informal event, and that's how it went. We rolled with questions, shifted the agenda as needed, and ate most of the snacks. @rebolek deserves a big round of applause (and leftover beer) for arranging the facility and much more. As is often the case, small groups clustered during breaks to talk about specific topics and get to know each other. </div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
We were able to answer questions some of the newcomers had, both verbally and in code. It's great to have someone ask "How hard would it be..." and be able to sit down and live code a demo. This is the power of Red.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
As far as planned presentations, Nenad gave an overview of Red, and showed some C3 examples, which led to Q&A on more general blockchain aspects as well. It was a nice mix of people on the Red and blockchain sides, helping each other learn a bit about the other side. @rebolek demoed his `values` structured editing prototype, which was very cool. If you thought you needed fancy GUI editors to offer help and extra information to users, you haven't seen what can be done with unicode chars and a little (or a lot of) creativity. Gregg gave a very quick rundown of possible approaches we can take to offer a cron type system for C3. If you aren't into the blockchain side of things, you may not know that there is no internal cron, no way to run something on a schedule. All triggers must come from outside the blockchain, as there isn't even the concept of a clock or current time in the EVM. Those things are handled via "oracles", which is just a fancy term for a reference to some higher authority you trust, outside the blockchain.<br />
<br />
The slide deck for Red/C3 presentation can be found <a href="https://static.red-lang.org/C3/Red-C3.pdf" target="_blank">here</a>.<br />
<br />
Thanks to @endo64 for bringing a camera. As you know, getting a good video setup is a job unto itself, and we couldn't arrange for a separate person to do that this time, but with simple raw footage and some magic applied by @x8x, you can see much of what happened below. Some of the video is hard to read, but many of the graphics are available elsewhere, and we'll make slides available soon.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture" allowfullscreen="" frameborder="0" height="360" src="https://www.youtube.com/embed/qRFX9pQd22w" width="640"></iframe>
</div>
<br /></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicHOnYFLFUfqhn_ybu8fXC89iaK5MFy_nZXIImyva_2pBRQ25nPLHhcR0mGeo3d8jvrVR0DJM-40L6WwSLPOelLu0J_G0Wbi3plkwFEqgwDc2BR49Ny57QIIh3S3sx6EVsXnp9_6uZSmQ/s1600/red-con-people.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="612" data-original-width="816" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEicHOnYFLFUfqhn_ybu8fXC89iaK5MFy_nZXIImyva_2pBRQ25nPLHhcR0mGeo3d8jvrVR0DJM-40L6WwSLPOelLu0J_G0Wbi3plkwFEqgwDc2BR49Ny57QIIh3S3sx6EVsXnp9_6uZSmQ/s640/red-con-people.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
A few RedCon attendees</div>
<br /></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<h3>
Red Foundation Meeting</h3>
</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
Another highlight from the trip was a meeting with François Jouen and Azouz Guizani from the Red Foundation. It was a 5 1/2 hour meeting, though we were in Paris, so much of it occurred over a meal in a small cafe. That may sound appealing, but when you forget your notebook, the napkins fill up quickly.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
François has been a strong advocate of REBOL and Red for many years. His teaching position and deep desire to help his PhD students led to development of a curriculum where Red is a requirement, not an option. Students have 8 3-hour sessions where they learn the fundamentals of the language, and how to apply it to their domain. These are not Computer Science PhD candidates, but art historians, literature majors, and cognitive scientists. They aren't taught Red in order to make them programmers, but to enable them to use the computer as another tool in their arsenal. </div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
The passion for his work, and the joy he clearly gets from using Red, are inspiring and infectious. François' combination of skills and talents aren't something we can clone, but seeing it first-hand led to discussion of what it might take to reproduce his approach in other places. You need a champion, and a mix of technical, leadership, and teaching skills; plus deep domain knowledge. François has ported almost half of the OpenCV computer vision library (~400 functions) to Red, as RedCV.<br />
<br />
His demos cover a lot of ground, run fast, and demonstrate the value software can bring, if we make it more accessible to people. Some of his students have tried to program before, because they know that's the future in almost any science, but it's too painful and too far from what normal people can do. But when his short course shows them how to create a GUI with VID, they feel empowered. They can build simple tools and see results. With RedCV, an art historian can load an image of a painting, then create their own pipeline of filters and convolutions, to bring out hidden details that are invisible to the naked eye.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
Flush with the excitement of seeing Red in use, in the real world, we walked to a small cafe nearby and spent the next 4 hours discussing goals for the Foundation, explaining more about the blockchain aspect and RED tokenomic possibilities to François and Azouz. We also talked about their specific uses cases and needs that might be solved with Red. There are so many things to do, and we sometimes feel overwhelmed, but when you sit down with other people who believe in what you're doing, and support you, it makes all the difference in the world. The next night Azouz joined us again, along with some old colleagues of Nenad's, and we talked more about the technical side.<br />
<br /></div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<h3>
Conclusion</h3>
</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
The days were long, and the nights short. The exhilaration of collaboration gave way to exhaustion as we headed back to our hotels, finding it hard to stop talking and planning. It all came and went quickly, but we learned a lot.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
While we hoped for a different experience at DevCon, we have to engage the community if we want to bring value to it. The high points of our meetings with other passionate teams and individuals made it worthwhile. Seeing old friends, and meeting new ones, a room full of Team Red, was great. The little time we had to see sights was invariably filled with tech chat. That all ended too soon and we jetted to our next locations. More plans are in the works, and we'll pick up speed again once the lag wears off.</div>
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
Until then, Happy Reducing!<br />
<br /></div>
<hr />
<div style="line-height: 1.38; margin-bottom: 6pt; margin-top: 18pt;">
<a href="https://www.blogger.com/null" name="footnote-1"><sup>1</sup></a> A comment on the "BUIDLer" term. The Ethereum community wants to grow, to gain adoption, to see their tool used in the real world. We have many thoughts on that, but if that's what they really want, the first thing many projects need is to take themselves seriously. That may sound harsh, and there are serious people doing real work, to be sure. But when billions of dollars are in the mix, and at risk (look how much has been lost this year alone, through hacks and mistakes), it is irresponsible not to take that seriously, and to use inside jokes and memes publicly and prominently. Adding a hashtag and memetic baggage to a powerfully simple concept such as “build” doesn’t mean doing anything actionable or making something real, other than smoke and mirrors. It also puts projects like ours in a difficult position. We are serious, and want to be viewed that way, but our choice to not bury our work in jargon sets us apart. They value inclusivity, so we hope they'll consider our approach and views as another type of diversity: one that evaluates the project on merit, not its use of trendy jargon or rave afterparties.<br />
<div>
<br /></div>
<div>
<div>
<a href="https://www.blogger.com/null" name="footnote-2"><sup>2</sup></a> We see a widening gap in the field between those building the technology, and those on the speculation side, who don't know or care much about the technology. While this problem has become endemic to the blockchain, crypto, and distributed-ledger-technology world, it becomes ever more evident at each subsequent major industry event. The majority of attendees of late aren't business people, in the sense that we think of business, but they can throw buzzwords around like you wouldn't believe. We met CTOs who didn't seem to know much tech, and got vague descriptions of what people did. "We onboard people to the blockchain," or "our project decentralizes processes and devaluates [sic] intermediaries." When your own most basic introductory explanation of your business function itself needs a translation to plainspeak, you don’t actually have a job description; without clearly defined roles and results, your whole project is nebulous. If the Ethereum Foundation, project founders, and the community really want to see Ethereum succeed, they themselves need to “devalue intermediaries” spouting what sound like a hollow string of buzzwords with no connection to the real world.</div>
</div>
<div>
<br /></div>
</div>
Unknownnoreply@blogger.com2