Advanced Distributed Computation

SpectroDynamics LLC

Advanced Distributed Computation

Butterfly^TM

Butterfly^TM is an advanced distributed computation platform that SpectroDynamics has invented and developed. It is architected on top of Common Lisp (i.e. the most powerful programming language ever conceived) as an effortless message-passing protocol that allows the easy and scalable creation and management of any number of cooperating & inter-communicating computational processes.

Processes can be spawned and connected in a distributed fashion, just as easily as they can be made to run on one and the same node. The application code can choose to remain oblivious about the actual location of cooperating processes.

Distribution networks can be as simple as a point-to-point link, a mesh, or a LAN, or they can be as wide and elaborate as a corporate intranet or even the Internet, itself. Software written on the Butterfly^TM platform supports hot-swapping of code updates over secure network connections.

This has some rather revolutionizing ramifications in terms of new possibilities of system functionality. For example, field-service of deployed systems, where traditionally technicians have to be dispatched to remote locations, to upgrade, or service equipment, etc. can now be done remotely, safely and economically. Software developed under Butterfly^TM can also be made to interface with appropriate heuristic decision engines and corresponding logic production-rule systems so as to make a whole system become self-healing, thereby even further enhancing its resilience, robustness, stability, and around-the-clock service.

Butterfly(TM) runs under OSX, Windows (Vista or XP) and Linux operating systems.

It is completely portable on industry-standard Common Lisp platforms such as LispWorks, Franz's Allegro CL, ClozureCL, and SBCL.

WHAT IS IT?

Butterfly^TM is an elaborate set of Lisp libraries, which enables the straightforward development of coordinated, modular, and concurrent computational processes. These processes differ from traditional operating system threads. They can but do not have to be inter-related, they can be synchronized and they run in parallel or sequentially on the same host computer system or they can be distributed on a multitude of machines that are connected in any possible networking arrangements.

Application development does not have to be done in Lisp (although doing so would undeniably produce a very powerful application) .The foreign function interface (FFI) provided by Lisp as well as the API that is provided by SpectroDynamics allow one to develop such applications using any major other language, e.g. C, C++, Java, etc.

WHY LISP?

It is not a coincidence that Lisp is considered the "Queen of Languages". Lisp started as the lingua franca of the artificial intelligence computing and symbolic processing communities in the 50's and all the way into the 80's, but it has evolved by now to become the most powerful mathematical formulation of a super-programming language with unprecedented plasticity. It is therefore known in the research communities as a language-building language.

Lisp enables the development of code that significantly follows the functional programming paradigm. Software modules are conceived and structured as inter-related mathematical functions. They receive functional arguments e.g. as input parameters, and upon completion they return appropriate values, which in turn feed other modules and so forth. This approach has been arguably shown to be the best path towards mathematical correctness and verifiability of code, especially when accompanied by strong-typing practices.

Lisp also enables the choice between strict evaluation and so-called lazy evaluation, each one with its own set of flexible trade-offs, which allow the programmer to tune a solution to perfectly fit the problem at hand.

Last, but not least, Lisp allows experimentation by the programmer at the highest possible levels of abstraction liberating him or her from the headaches and nightmares of mundane trivia and dangerous concepts as managing memory pointers. The programmer therefore spends more time thinking about the actual problem at hand and not about how to make the computer's internal details behave accordingly. High-levels of abstraction also minimize the number of code lines needed to produce a complete, robust, non-verbose, and elegant solution.

There is a lot of research and it is confirmed by our own experience that a good Lisp program of a certain length (e.g. measured in lines of code) will accomplish much more than what an alternative language implementation (i.e. C++, Java, etc.) will at best only come close to, and at 15-20x the lines of code, effort and cost. Leaning on today's world-class Lisp-compiler technology, Lisp dramatically enhances programmers' productivity, and not just the capabilities of their software.

Most spectacularly, Lisp treats data as code and code as data. Lisp is the language par excellence that allows one to write software-writing software. Lisp completely blurs the boundary between read-time, compile-time and runtime. If a piece of running software, as part of its normal execution, can also be made to write its own new piece of software (or, rewrite even itself, in a self-morphing scheme), it can also easily call the Lisp compiler at run time as an ordinary subroutine to work on the automatically produced source code thereby producing on-the-fly new executable code that can be put to work immediately on the spot.

No other programming language comes even close to the possibilities that Lisp enables. It is not a coincidence that world-class organizations like NASA, DARPA, etc. depend on advanced Lisp code for the delivery of software applications with extraordinary capabilities and robustness.

If you are interested in advanced Lisp-based development services or special assistance from SpectroDynamics, please click on the following link:

Services

HOW IT WORKS

Butterfly^TM builds exclusively on the underlying Lisp environment and behaves at the user level in a way that is analogous to the workflow of application software written in the Erlang programming language. However, by the mere fact that Lisp is an undeniably much more powerful language than Erlang, there are numerous differences between the two worlds.

BUTTERFLY(TM) PROCESSES & SECURITY

Butterfly^TM offers several primitives, which in a very straightforward way allow the creation, inter-linking, and graceful termination of concurrent processes. Every process is characterized by a unique PID (process identification) number that is used for inter-process communication. Butterfly^TM offers another series of primitives that handle the sending and receiving of messages between processes. Message-passing is the underlying method of scalable cooperation between processes. A whole heavy-duty cryptographic infrastructure and methodology is deployed on top of this communication scheme to ensure that no eavesdropping, misauthentication, spoofing, playback, or other illegitimate access to the link can occur while inter-process streams traverse an open network. Terminated processes signal their end to their controller process and transmit accordingly crucial information on their state before their termination. This helps ensure the creation of stable and resilient code that offers higher levels of reliability.

INTER-PROCESS COMMUNICATION

Every Butterfly^TM process has its own sophisticated mailbox where incoming messages from other processes are enqueued while awaiting its attention. A process has multiple flexible ways to sort out the contents of its mailbox based on sophisticated pattern-matching criteria, which enable the prioritization of dealing with incoming messages. Some messages simply carry information between processes, while some other messages enable the remote execution of code using the approach of remote-procedure calls (RPC). A hierarchical world based on supervisor processes and worker processes further enables the layering and compartmentalization of modular responsibilities. This helps create harmoniously cooperative code that can be easily mapped onto multiple processors.

A very good example of the extraordinary design flexibility afforded by the Butterfly^TM platform is the Okeanos^TM multi-user distributed object-oriented database from SpectroDynamics. For more information about it, please feel free to click on the link here below:

Okeanos(TM) - Distributed Object-Oriented Database

IN CONCLUSION

If you have an application that can be best structured (or restructured) into a concurrent computing environment that runs either on one or on many distributed computer systems, then you must consider Butterfly^TM. If you have in-house Lisp expertise, SpectroDynamics will train them to make the most out of a license of the Butterfly^TM platform. If you don't have access to your own Lisp expertise, SpectroDynamics will be happy to provide you a one-stop solution. We not only license the underlying Butterfly^TM technology, but we also help companies by developing and delivering world-class customized solutions for them using these cutting-edge technologies.

Please contact us so we can discuss your needs and provide you with suggestions as to how to address them.

info AT spectrodynamics DOT com