async/.await

Ok, so I’m exploring this reliable-UDP publish/subscribe system, but taking full advantage of the asynchronous promises (ha.. ha..) of smol, async-std or tokio. It is essentially an oversimplified re-imagining of RTPS, which is used in DDS to transmit commands in battleships.

The main objectives of this project are:

1. Entirely automatic and robust discovery.
2. Publishers and subscribers on any device in the network.
3. As little configuration as possible.
4. As fast as possible.

Each device runs exactly one Participant, which is a process that handles the administration of Publishers and Subscribers, which are also processes.

Publishers send messages to a specific topic:

let publisher = Publisher::new("/hello").await;
:
publisher.send(&message).await;

Subscribers handle incoming messages from a topic:

fn on_message(message) {
    // process the incoming message
}

:
let subscriber = Subscriber::new("/hello",on_message).await;

Discovery between the participants, and making sure the publishers are aware of all related subscribers in the network is completely transparent to the user.

The actual data is sent over a reliable UDP mechanism, more on this later.

Ideally, when publisher and subscriber are on the same device, this system would utilize shared memory to not even transmit the message at all. More on this later also.

Also, ideally, serialization and deserialization of the messages is extremely fast. It seems that serde (or similar) and Rusts derive macro system is extremely well suited for this. RTPS (or related Protocol Buffers, gRPC, etc.) uses a separate domain language and separate compilation step to describe the messages, but we don’t need to.

I tried a project like this before in C, a very long time ago. The hard part then was keeping track of an enormous number of open sockets, as well as the explicit state management for each connection and messages being transmitted. It took several weeks to get somewhat off the ground, and hunting down bugs in raw C socket code is not the most relaxing of times.

Clearly here asynchronous programming comes to the rescue, and I’m hoping that it will take considerably less time to get to any proof-of-concept.