February 14, 2019

January 2019 Update



Stepping-up


2019 started with continued advances in the main development branch, implementation of features, and triage of pending tickets. For example, smart merging of a style's actors with the actors of a particular face instance was introduced in the VID dialect. Example:

view [
    style my-button: base
        on-down [face/color: face/color / 2 do-actor face event 'click]
        on-up   [face/color: face/color * 2]

    my-button "Say hi!" red on-click [print "hi"]
]

But this is just a tiny drop in the January ocean, and nothing special in the grand scheme of things. More than 50 commits to the mainline, lots of doc updates, and many improvements to the Red Wallet, which we'll talk about in another blog post. On the other hand, Red's progress is rarely trivial and silky smooth, and this write-up gives a deeper insight into our daily routine. Well, what did you expect, Pumpkin Juice? ;-)

Alas, as much as we love coding, organizational heavy-lifting needs to be done too — one of our short-term priorities this year is relocation of core team to Europe, and (ongoing) activity on this front took a lot of our time and resources in January. That is to say, after little overture, the core team's main activity shifted behind the scenes, giving the stage to our outstanding community's efforts. 1

Community highlights

GTK backend


Red strives to be a truly cross-platform language, but the project's roadmap always implies a certain order of priorities, where some milestones come later than others. One such missing cross-platform feature is the View engine's backend for Linux — its development requires dedicated resources, which the core team cannot fully afford, probably until near the 1.0 release.

With that in mind, it is much to our delight that development of GTK backend is being carried out as a dedicated community effort. We'd like to take an opportunity and thank all the people participating in it — their proactive contributions make Red a truly open-source project, and help us to advance it towards our collective vision. Big thanks to @rcqls for taking the lead on this.

Last but not least, you can track development's progress in a recently created GTK room. Don't forget to visit it and give our contributors all the praise they deserve! Or even better — become one of them and get a share of fame and glory! ;-)


View and Rich-text tweaking


Not only do missing parts of the View engine get developed, but existing ones receive thorough maintenance too. Features and fixes.

Among fixes are base and text rendering, Red's codebase received a contribution for testing of image-related features on Windows platforms, accompanied by more than 300 tests. Coupled with our "null" testing backend (covered in some detail in here) this casts a wide net for catching regressions in View.

After the 0.6.4 release announcement, many users checked out rich-text support and gave Red's Rich-text dialect a proper spin, with numerous bug fixes and tweaks being submitted in the process.


Editing, with a capital E


As you can see from the above, native GUI support is one of our strongest aspects, which piques the interest of many community members.

Another special feature that catches everyone's fancy is Red's "code as data" nature. But, while giving Red enormous expressive power (metaprogramming and the creation of domain-specific languages, DSLs for short), it makes some things trickier — with the boundary between code and data being blurred, and multiple dialects (our term for embedded DSLs) at one's disposal, it is extremely challenging to assign any particular meaning to a given piece of code (data!); both for humans (esp. newr Red programmers) and automatic tools, such as syntax highlighters, static code analyzers, debuggers and linters.

Historically, this was the main argument when it came to Rebol's tooling. Sprinkling your code with print and probe is still the most common debugging method, and error reporting boils down to stack traces rather than source text info; text editing support can accommodate only cosmetic niceties like "keywords" highlighting and indentation, covering only general subset of the language.

Being a direct descendant of Rebol, we carried over this bag of concerns, aiming to resolve them at some point in development. A quite natural answer to points raised would be an editor written in pure Red and its dialects, integrated in our self-contained toolchain.

And while writing a basic version of such editor is rather trivial, such a naive approach leaves much to be desired, and keeps questions about proper tooling surrounded with an air of uncertainty, begging for a detailed investigation. Thankfully, this is where community projects and experiments come to the rescue!
  • @rebolek's Values, announced at last year's RedCon, is a sophisticated projectional editor with Vim's flavor and an ingenious schtick — rather than working with textual form, it loads a program and directly manipulates its internal data structure (a block! of values), reflecting all changes back in the text area.

    Idiomatically written, and offering sheer power in a few keystrokes, it struck us in awe back at RedCon, and stimulated a lot of technical discussions. With our new GUI console's plug-in API on its way for 0.6.5 release, we are eager to see how it will get integrated in day-to-day user experience, and what fresh insights it can bring to all of us.
  • @toomasv is well-known in our community as an avid Draw and View tinkerer, and his recent experiment only firms this reputation.

    Datatype-aware syntax highlighting, contextual help, auto-completion, step-by-step evaluation and bunch of other niceties — all in ~1K lines of code! Magic, you say? No, Reducing at its finest. ;-)


Projects like this (and all others that we didn't get a chance to mention) aren't just eye-candy for the public — they help us, core developers, to evaluate all options and to revisit old ideas in a new light. Besides, it's extremely rewarding to see Red (in its alpha stage) being put to real use.

Once again, our cheers go to all the people who dedicate their time and efforts to support us. No matter how small your contribution, it makes Red stronger and readier for the prime-time: day by day, bit by bit.

C3 project


Since the core team relocation is our ongoing No.1 priority, development of the C3 project got sidetracked a bit. There are still a few final touches that need to be done (e.g. finishing AST parser, hex! and bigint! datatypes support) but, aside from that, the first alpha version of compiler is almost ready for a public release. The first 90% of a task takes 90% of the time, the last 10% takes the other 90%.

In the meantime, we have one appetizer to serve: RedCon talk about C3! Due to technical issues and time constraints, it became available only recently, and was captured on a nearby web-camera. Because of that, video quality and recording angle are so-so, but the good news is that you can gain a deeper insight into the blockchain aspect of Red... and witness the talking belly. :-)




As evident from the talk2, C3's design proved to be much more challenging than we initially anticipated, muddled by pesky EVM (Ethereum Virtual Machine) constraints (like gas), lack of good reference documentation, complex tooling and blockchain hype haze.

At this point, we're convinced that at least two layers are required for C3 to be a sound smart-contract development solution:
  1. The low-level one, named C3/System, is a language with simple semantics (close to Solidity, but without foot-shooting), serving as a base for building high-level dialects on top of it. We also want to implement a couple of experimental features, like off-chain Cron-like scheduler of Smart Contract execution and embedded FSM (finite-state machine) sub-dialect.
  2. Possibly multiple high-level layers, each covering different blockchain aspect and tailored to specific solutions: creation of crypto tokens, gambling, in-game collectibles, legal contracts and business logic, bids and auctions, asset tracking... who-knows-what people may come up with.
Despite all the media craze surrounding blockchains, the technology itself is still in a nascent state, constantly searching for a real-world use-case. Because of that, navigating this new and constantly shifting space (not to mention building a development platform like C3) is like sailing in stormy weather.

In light of the above, we owe much to @9214 for his help with C3 preliminaries and conducting in-depth research on Smart Contract languages. His constant background presence affirmed many of our assumptions, and guided us past many reefs and pitfalls.

Back in the day, our "Red goes blockchain" announcement generated much controversy, and prompted many people (both community members and outsiders) to cast their doubts on this decision. Some go as far as to dismiss the Red project entirely, based on our new roadmap.

Yet, very few realize that, in some sense, we're picking up the game that Rebol left — with Carl's conception of X-Internet3 replaced by decentralized consensus-driven P2P networks, and EVM instead of IOS. Indeed, Smart Contracts are just very limited...slow Reblets/services in disguise ;-) And behind blockchain fatigue lies the brave new world, where Red can be the missing puzzle piece and stepping stone towards simple, human-friendly tooling.

All in all, we hold to our beliefs, and going to play this card, aiming to hit the jackpot. But we're also not putting all our eggs in one basket. Bear with us as we tread forward, and stay tuned for future updates!

Wrapping-up


January, with its holidays and family reunions, is nothing but a calendar sheet. The work on Red never ceases, spans through weekends, occupies all time and space (both physical and mental) and now reaches across the globe. For some, it became the animating principle of their lives.

With our hard work at stake, and with 1.0 release in near sight, encounters with the outside world get more frequent and progressively challenging — criticism and cross-checks with reality are important, but it's equally important to apply a fair yardstick, and judge the project on its own merits.

That's why, when it comes to media coverage and public discussions, we expect outsiders to stay open-minded and not to cast aspersions lightly. We want Red to succeed, and welcome technical debate and comparisons with other projects (like, e.g., Red vs. Racket; again, kudos to @9214 for writing such a well-articulated response). But, more often than not, discussion threads turn into exercises at damage control, with constructive points being hamstrung and torn to pieces in a cross-fire.

Our mission to fight software complexity is challenging, and not for the faint-hearted. Remember — the best way to prove one's case is not in word, but in deed... and the strongest argument is a tech-savvy pull-request. ;-)

Come to think of it, I can't help remembering Pierre Rabhi's tale:
Once upon a time, the scorching fire swallowed the forest. All the animals, terrified and powerless, watched it destroying their homes from afar. Only little Hummingbird kept flying back and worth between the fire and nearby pond, carrying tiny drops of water in its beak and throwing them in the flames.  
Irritated by bird's constant flickering, grouchy Armadillo yelled: "Hummingbird! Don't you see that your pathetic attempts won't save the forest?"
"I know.", answered Hummingbird, "but I'm doing my part."

Hectic 2018 barely passed by, but we're already steering wheels on our new roadmap. There is more news to announce, once the dust has settled. We'll be in touch. Until then, Happy Reducing!

-- Post by 9214, edits and amendments by Gregg


Footnotes

  1. Remember that you can track progress of Red projects (both community-driven and official ones) at https://progress.red-lang.org/
  2. Reminder: C3 presentation slides in PDF format are available here.
  3. "X" here stands for eXecutable and eXtendable.

January 6, 2019

Full steam ahead!

The past year

It has been a tough year 2018, filled with roller-coasters and instability. The constant dropping of the crypto markets has forced us to change our plans pretty much every three months, in order to cope with that market uncertainty and our diminishing means. Though we have taken measures to ensure the proper funding for Red project for the next few years, so no worries on that side. All that has slowed down work on Red's core and even our new C3 language. That said we still have managed to make some great progress in the past year, e.g.:

  • RED Wallet released: a good showcase of what can be achieved in a small codebase using a truly full-stack programming language like Red, from USB and hardware keys drivers to blockchain network interfacing and cross-platform UI!
  • Garbage Collector: a simple mark & sweep, compacting GC for Red! That is a big step towards a completed production-ready Red version 1.0.
  • Red console fully re-implemented in Red itself, allowing new features like syntactic coloring and user plugins.
  • New rich-text display widget and related RTD dialect.
  • DPI-independent support for Red/View.
  • Preliminary port! and bigint! datatypes implementations.
  • eth:// port for interacting with the Ethereum blockchain (to be released publicly very soon).
  • 350 tickets closed, among which 136 are bugfixes.
  • RedCon1 held in Prague in November!
  • Extended Red team with now about 12 people in total.

We are also working on structuring the Red Foundation and have a new recruit working on tokenomics. So the tokens retro-distribution to kick-start tokenomics should happen very soon.

The plan for 2019

We are starting the year with big targets as we need some last big pieces of the Red project to be implemented to reach both our technical and business goals. Here are the main tasks we will handle during this new year:

  • C3 toolchain release: the C3/System language spec was defined last October and work on the compiler and related toolchain is on-going. This is our main short-term priority as we want to test our business options in the smart contract market while it is still in its early days.

  • Red full async I/O support (0.7.0 milestone): we already have a PoC for async tcp/udp support, with Rebol-like event handling API. There is still a need for more design work (on-going right now) in order to provide the best possible API for Red users. For now, we are trying to avoid going full-force into the deep territory of defining a general concurrency support (it's scheduled for 0.9.0 milestone). I'm not yet sure if we can escape stepping into it anyway if we want to go beyond the Rebol async model.

  • Full Android support for Red: we have a preliminary implementation that has proven to be very promising! With the integration of async I/O and ports, we will be able to complete the wrapping of the Android API using decent human-friendly abstractions. We need to advance Android support to at least beta level by the end of this year. This is one of the major goals we need to achieve.

  • Red/Pro: it is coming this year! It will be our first commercial product and set of online services targetting both individual developers and enterprises. It will include a new backend for Red's toolchain, providing a state-of-the-art optimizing layer and support for many new platforms, including 64-bit ones (though full 64-bit Red support will probably be for next year). That new backend will provide an alternative to our current code emitter and linker, the rest of the toolchain remaining the same. The current Red toolchain will be known as the "community" version, and continue being the main development branch.

In addition to all that, a 0.6.5 milestone for Red is in the plans, aiming at making the Red console the main focus for Red users by providing:
  • prebuilt versions of the Red console.
  • ability to invoke the Red toolchain from the Red console.
  • a plugin API to extend the console in many ways.

In order to better focus on these tasks and better interact with our new team members in Europe, we will be relocating our core team from different parts of the world to Europe (Red Foundation is located in Paris). We also plan to make a RedCon2 developer conference in Europe later this year.

Many thanks to all the people who have contributed or helped in any way with advancing Red and managing the community. How do we climb the tallest mountain? One step at a time! And the top is in sight now, we're almost there. ;-)

Happy New Year to all, and keep up the great work you are all doing!

Fork me on GitHub