Skytable 0.8.2: BlueQL features, pipelines, SE ops, fixes

We’re extremely happy to announce the release of Skytable 0.8.2. This release includes some new additions to BlueQL, improvements to storage engine operations and fixes a bunch of bugs. Let’s take a look at what has changed.

New BlueQL features

Upsert

We added the UPSERT query that enables you to either replace a row if the row is present or insert a new one. This can be extremely useful in cases where you want to run SET like K/V operations and the current value of the row isn’t important. The syntax is largely the same as BlueQL’s INSERT:

UPSERT INTO myspace.mymodel('hello', 'world') # parameters omitted

Query Shorthands

We’ve also added support for query shorthands that can come in clutch if you’re trying to reduce network overhead or if you’re debugging. These are the query shorthands:

• INS: INSERT
• SEL: SELECT
• UPD: UPDATE
• DEL: DELETE
• UPS: UPSERT

Pipelines

Pipelining support has finally returned to Skytable. You can now send multiple queries to the server at once and have the server execute them serially and return responses in the order queries were sent. Here’s an example using the Rust driver:

use skytable::{pipe, query, ConnectionAsync, Config};

let mut con = Config::new_default("root", "pass").connect_async().await.unwrap();
let pipeline = pipe!(
    query!("create space db"),
    query!("create model db.mdl(name: string, fav: float64)"),
    query!("insert into db.mdl(?,?)", "sayan", 3.14_f64)
);
// use the results this somewhere
let _ = con.execute_pipeline(&pipeline).await.unwrap();

Storage engine operations

We have made several improvements and fixed several bugs in the storage engine. The most important addition is the ability to explicitly trigger a compaction, where the database files are optimized to smaller sizes (if possible). This can be invoked by running:

skyd compact

The server now also attempts to automatically compact the database files during startup. Note that the server will not forcefully compact the database (unlike skyd compact which triggers a forceful compaction); instead, the server will determine if a compaction is needed using internal heuristics.

Overall, these utilities help you as a database administrator to ensure that database files don’t grow too large and also helps improve performance and memory usage (due to smaller database files). Before any compaction is run, all database files are fully backed up in the backups/ directory so in the event something goes wrong, you can always restore your files from the backup.

ARM64 releases

As announced earlier, from this release (and onwards) we are shipping full bundles of ARM64 binaries for macOS and Linux targets. The Linux target also includes a Debian package which sets up systemd units, passwords and more.

Looking ahead

We strongly recommend you to upgrade to Skytable 0.8.2, especially because of the SE compaction system that was added (even if you do not plan to use the new BlueQL additions). Go ahead and download the latest release here. As always, if you find any bugs please report them here.

We’re currently hard at work on the clustering and replication system and we hope to ship them soon. See you at the next release!