cryomongo

A MongoDB driver written in pure Crystal.

This is a temporary fork to update the codebase while working on performance, stability, and compatibility with the latest MongoDB LTS release (8.0). Merging to the upstream original repository is part of the plan, as soon as the work on updating is concluded.

It follows the newest MongoDB specifications (MongoDB 8.0).

On a technical level, the current update already includes:

MongoDB 8.0 & Spec Compliance: Bumped to the latest wire version of 25. Implemented the official MongoDB Unified Test Format (UTF), already 100% compliant with the official crud, retryable-reads, and retryable-writes test suites.
Performance & Reduced Allocations: Optimized TCP socket, replacing intermediate buffer allocations with read-only IO::Memory byte scanning. Optimizations for efficiency, performance and less GC pressure, like lock-free atomics, single-pass iterators and user-space pipes (jgaskins/pipe for GridFS).
Crystal 1.20+ Ready: Updated for the latest Crystal 1.20/1.21 branches, migrating to the updated Sync::Mutex, replacing Time.monotonic with Time.instant, and full support for the new Parallel Execution Contexts.
BSON 8.0 Integration: Uses the updated alumna/bson.cr shard, enabling zero-allocation Extended JSON, native 16-byte Decimal128, and the new Binary Vector types.

Roadmap & Checklist

The update roadmap is actively being worked on. Below is the current progress against the official specifications:

Unified Test Format (unified-test-format.md): Modern engine built to run the official declarative JSON test files.
CRUD Operations (crud.md): 100% compliant.
Retryable Writes (retryable-writes.md): 100% compliant. Gracefully handles network failures, step-downs, and errorLabels.
Retryable Reads (retryable-reads.md): 100% compliant.
Transactions (transactions.md & transactions-convenient-api.md): 100% compliant. Implements robust exponential backoffs and strict timeouts.
Sessions & Causal Consistency (causal-consistency.md): 100% compliant. Full support for snapshot reads and causally consistent write propagation.
Server Discovery and Monitoring (server-discovery-and-monitoring.md): Up next.
Authentication (auth.md): Pending SCRAM-SHA-256 validation.

Cryomongo is a high-performance MongoDB driver written in pure Crystal. (i.e. no C dependencies needed.)

Compatible with MongoDB 8.0+. Tested against: 8.0.

⚠️ BETA state.

If you are looking for a higher-level object-document mapper library, you might want to check out the moongoon shard.

Installation

Add the dependency to your shard.yml:

dependencies:
  cryomongo:
    github: alumna/cryomongo

Run shards install

Usage

Minimal working example

require "cryomongo"

# Create a Mongo client, using a standard mongodb connection string.
client = Mongo::Client.new # defaults to: "mongodb://localhost:27017"

# Get database and collection.
database = client["database_name"]
collection = database["collection_name"]

# Perform crud operations.
collection.insert_one({ one: 1 })
collection.replace_one({ one: 1 }, { two: 2 })
bson = collection.find_one({ two: 2 })
puts bson.try(&.["two"]) # => 2
collection.delete_one({ two: 2 })
puts collection.count_documents # => 0

Complex example with serialization

require "cryomongo"

# We take advantage of the BSON serialization capabilities provided by the `bson.cr` shard.
record User,
  name : String,
  banned : Bool? = false,
  _id : BSON::ObjectId = BSON::ObjectId.new,
  creation_date : Time = Time.utc do
  include BSON::Serializable
  include JSON::Serializable
end

# Initialize Client, Database and Collection.
client = Mongo::Client.new
database = client["database"]
users = database["users"]

# We set majority read and write at the Database level.
database.read_concern = Mongo::ReadConcern.new(level: "majority")
database.write_concern = Mongo::WriteConcern.new(w: "majority")

# Drop and recreate the Collection to ensure that we read later only the documents we inserted in this example.
{ Mongo::Commands::Drop, Mongo::Commands::Create }.each do |command|
  database.command(command, name: "users")
rescue e : Mongo::Error::Command
  # ignore the server error, drop will fail if the collection has not been created before.
end

# Insert User structures that are automatically serialized to BSON.
users.insert_many(["John", "Jane"].map { |name| User.new(name: name) })

# Fetch a Cursor pointing to the users collection.
cursor = users.find

# Iterate the cursor and use `.of(User)` to deserialize as the cursor gets iterated.
# Then push the users into an array that gets pretty printed.
puts cursor.of(User).to_a.to_pretty_json
# => [
#   {
#     "name": "John",
#     "banned": false,
#     "_id": {
#       "$oid": "f2001c5fb0a33e0264e2ea05"
#     },
#     "creation_date": "2020-07-25T09:52:50Z"
#   },
#   {
#     "name": "Jane",
#     "banned": false,
#     "_id": {
#       "$oid": "f2001c5fb0a33e0264e2ea07"
#     },
#     "creation_date": "2020-07-25T09:52:50Z"
#   }
# ]

Features

CRUD operations
Aggregation (except: Map-Reduce)
Bulk
Read and Write Concerns
Read Preference
Authentication (only: SCRAM mechanisms)
TLS encryption
Indexes
GridFS
Change Streams
Admin/Diagnostic commands
Tailable and Awaitable cursors
Collation
Standalone, Sharded or ReplicaSet topologies
Command monitoring
Retryable reads and writes
Causal consistency
Transactions

Conventions

Methods and arguments names are in snake case.
Object arguments can usually be passed as a NamedTuple, Hash, BSON::Serializable or a BSON instance.

Documentation

The generated API documentation is available here.

Connection

require "cryomongo"

# Mongo::Client is the root object for interacting with a MongoDB deployment.
# It is responsible for monitoring the cluster, routing the requests and managing the socket pools.

# A client can be instantiated using a standard mongodb connection string.

# Client options can be passed as query parameters…
client = Mongo::Client.new("mongodb://address:port/database?appname=MyApp")
# …or with a Mongo::Options instance…
options = Mongo::Options.new(appname: "MyApp")
client = Mongo::Client.new("mongodb://address:port/database", options)
# …or both.

# Instantiate objects to interact with a specific database or a collection…
database   = client["database_name"]
collection = database["collection_name"]

# …or using `default_database` if the connection uri string contains a default auth database component ("/database").
if database = client.default_database
  collection = database["collection_name"]
end

# The overwhelming majority of programs should use a single client and should not bother with closing clients.
# Otherwise, to free the underlying resources a client must be manually closed.
client.close

# To enable SSL/TLS, use the `tls` option, alongside the `tlsCAFile` and `tlsCertificateKeyFile` options.
uri = "mongodb://localhost:27017/?tls=true&tlsCAFile=./ca.crt&tlsCertificateKeyFile=./client.pem"
ssl_client = Mongo::Client.new uri

Links

Authentication

Cryomongo only supports the SCRAM-SHA1 and SCRAM-SHA256 authentication methods without SASLprep.

require "cryomongo"

# To use authentication, specify a username and password when passing an URI to the client constructor.
# Authentication methods depend on the server configuration and on the value of the `authMechanism` query parameter.
client = Mongo::Client.new("mongodb://username:password@localhost:27017")

Basic operations

require "cryomongo"

client = Mongo::Client.new

# Most CRUD operations are performed at collection-level.
collection = client["database_name"]["collection_name"]

# The examples below are very basic, but the methods can accept all the options documented in the MongoDB manual.

## Create

# Insert a single document
collection.insert_one({ key: "value" })
# Insert multiple documents
collection.insert_many((1..100).map { |i| { count: i } })

# To track the _id, generate and pass it as a property
id = BSON::ObjectId.new
collection.insert_one({ _id: id, key: "value" })

## Read

# Find a single document
document = collection.find_one({ _id: id })
document.try { |d| puts d.to_json }

# Find multiple documents.
cursor = collection.find({ qty: { "$gt": 4 }})
elements = cursor.to_a # cursor is an Iterable(BSON)

## Update

# Replace a single document.
collection.replace_one({ name: "John" }, { name: "Jane" })
# Update a single document.
collection.update_one({ name: "John" }, { "$set": { name: "Jane" }})
# Update multiple documents
collection.update_many({ name: { "$in": ["John", "Jane"] }}, { "$set": { name: "Jules" }})
# Find one document and replace it
document = collection.find_one_and_replace({ name: "John" }, { name: "Jane" })
puts document.try &.["name"]
# Find one document and update it
document = collection.find_one_and_update({ name: "John" }, { "$set": { name: "Jane" }})
puts document.try &.["name"]

## Delete

# Delete one document
collection.delete_one({ age: 20 })
# Delete multiple documents
collection.delete_many({ age: { "$lt": 18 }})
# find_one_and_delete
document = collection.find_one_and_delete({ age: { "$lt": 18 }})
puts document.try &.["age"]

# Aggregate

# Perform an aggregation pipeline query
cursor = collection.aggregate([
  {"$match": { status: "available" }},
  {"$limit": 5},
])
cursor.try &.each { |bson| puts bson.to_json }

# Distinct collection values
values = collection.distinct(
  key: "field",
  filter: { age: { "$gt": 18 }}
)

# Documents count
counter = collection.count_documents({ age: { "$lt": 18 }})

# Estimated count
counter = collection.estimated_document_count

Links

Bulk operations

require "cryomongo"

client = Mongo::Client.new

# A Bulk object can be initialized by calling `.bulk` on a collection.
collection = client["database_name"]["collection_name"]
bulk = collection.bulk
# A bulk is ordered by default.
bulk.ordered? # => true

500.times do |idx|
  # Build the queries by calling bulk methods multiple times.
  bulk.insert_one({number: idx})
  bulk.delete_many({number: {"$lt": 450}})
  bulk.replace_one({ number: idx }, { number: idx + 1})
end

# Execute all the queries and return an aggregated result.
pp bulk.execute(write_concern: Mongo::WriteConcern.new(w: 1))

Links

Mongo::Bulk

Indexes

require "cryomongo"

client = Mongo::Client.new
collection = client["database_name"]["collection_name"]

# Create one index without options…
collection.create_index(
  keys: {
    "a":  1,
    "b":  -1,
  }
)
# or with options (snake_cased)…
collection.create_index(
  keys: {
    "a":  1,
    "b":  -1,
  },
  options: {
    unique: true
  }
)
# and optionally specify the name.
collection.create_index(
  keys: {
    "a":  1,
    "b":  -1,
  },
  options: {
    name: "index_name",
  }
)

# Follow the same rules to create multiple indexes with a single method call.
collection.create_indexes([
  {
    keys: { a: 1 }
  },
  {
    keys: { b: 2 }, options: { expire_after_seconds: 3600 }
  }
])

Links

Mongo::Collection

GridFS

require "cryomongo"

client = Mongo::Client.new
database = client["database_name"]

# A GridFS bucket belongs to a database.
gridfs = database.grid_fs

# Upload (using File.open ensures the file descriptor is closed automatically)
id = File.open("file.txt") do |file|
  gridfs.upload_from_stream("file.txt", file)
end

# Download
stream = IO::Memory.new
gridfs.download_to_stream(id, stream)
puts stream.rewind.gets_to_end

# Find
files = gridfs.find({
  length: {"$gte": 5000},
})
files.each do |file|
  puts file.filename
end

# Delete
gridfs.delete(id)

# And many more methods… (check the link below.)

Links

Mongo::GridFS::Bucket

Change streams

require "cryomongo"

# Change streams can watch a client, database or collection for change.
# This code snippet will focus on watching a single collection.

client = Mongo::Client.new
collection = client["database_name"]["collection_name"]

spawn do
  cursor = collection.watch(
    [
      {"$match": {"operationType": "insert"}},
    ],
    max_await_time_ms: 10000
  )
  # cursor.of(BSON) converts fetched elements to the Mongo::ChangeStream::Document(BSON) type.
  cursor.of(BSON).each do |doc|
    puts doc.document_key
    puts doc.full_document.to_json
  end
end

100.times do |i|
  collection.insert_one({count: i})
end

sleep

Links

Raw commands

require "cryomongo"

# Commands can be run on a client, database or collection depending on the command target.

client = Mongo::Client.new

# Call the `.command` method to run a command against the server.
# The first argument is a `Mongo::Commands` sub-class, followed by the mandatory arguments
# and finally an *options* named tuple containing the optional parameters in snake_case.
result = client.command(Mongo::Commands::ServerStatus, options: {
  repl: 0
})
puts result.to_bson

# The .command method can also be called against a Database…
client["database"].command(Mongo::Commands::Create, name: "collection")
client["database"].command(Mongo::Commands::Drop, name: "collection")
# …or a Collection.
client["database"]["collection"].command(Mongo::Commands::Validate)

Links

Concerns and Preference

require "cryomongo"

# Instantiate Read/Write Concerns and Preference
read_concern = Mongo::ReadConcern.new(level: "majority")
write_concern = Mongo::WriteConcern.new(w: 1, j: true)
read_preference = Mongo::ReadPreference.new(mode: "primary")

# They can be set at the client, database or client level…
client = Mongo::Client.new
database = client["database_name"]
collection = database["collection_name"]

client.read_concern = read_concern
database.write_concern = write_concern
collection.read_preference = read_preference

# …or by passing an extra argument when calling a method.
collection.find(
  filter: { key: "value" },
  read_concern:  Mongo::ReadConcern.new(level: "local"),
  read_preference: Mongo::ReadPreference.new(mode: "secondary")
)

Links

Commands Monitoring

require "cryomongo"

client = Mongo::Client.new

# A simple logging subscriber.

subscription = client.subscribe_commands { |event|
  case event
  when Mongo::Monitoring::Commands::CommandStartedEvent
    Log.info { "COMMAND.#{event.command_name} #{event.address} STARTED: #{event.command.to_json}" }
  when Mongo::Monitoring::Commands::CommandSucceededEvent
    Log.info { "COMMAND.#{event.command_name} #{event.address} COMPLETED: #{event.reply.to_json} (#{event.duration}s)" }
  when Mongo::Monitoring::Commands::CommandFailedEvent
    Log.info { "COMMAND.#{event.command_name} #{event.address} FAILED: #{event.failure.inspect} (#{event.duration}s)" }
  end
}

# Make some queries…
client["database_name"]["collection_name"].find({ hello: "world" })

# …and eventually at some point, unsubscribe the logger.
client.unsubscribe_commands(subscription)

Links

Causal Consistency

require "cryomongo"

client = Mongo::Client.new
# It is important to ensure that both read and writes are performed with "majority" concern.
# See: https://docs.mongodb.com/manual/core/causal-consistency-read-write-concerns/
client.read_concern = Mongo::ReadConcern.new(level: "majority")
client.write_concern = Mongo::WriteConcern.new(w: "majority")

# Reusing the original Mongodb example.
# See: https://docs.mongodb.com/manual/core/read-isolation-consistency-recency/#examples

current_date = Time.utc
items = client["test"]["items"]

# MongoDB enables causal consistency in client sessions by default.
# This is the block syntax that creates, ends and pass the session to collection methods automatically.
items_collection.with_session do |items|
  # Using a causally consistent session ensures that the update occurs before the insert.
  items.update_one(
    { sku: "111", end: { "$exists": false } },
    { "$set": { end: current_date }}
  )
  items.insert_one({ sku: "nuts-111", name: "Pecans", start: current_date })
  puts items.find.to_a.to_pretty_json
end

client.close

Links

Transactions

require "cryomongo"

# Initialize Client and Database instances.
client = Mongo::Client.new
database = client["db"]
collection = database["collection"]

# Create the collection.
{Mongo::Commands::Drop, Mongo::Commands::Create}.each do |command|
  database.command(command, name: "collection")
rescue e : Mongo::Error::Command
  # ignore the server error, drop will fail if the collection has not been created before.
end

# Set read and write concerns to perform isolated transactions.
# See: https://docs.mongodb.com/master/core/transactions/#transactions-and-sessions
transaction_options = Mongo::Session::TransactionOptions.new(
  read_concern: Mongo::ReadConcern.new(level: "snapshot"),
  write_concern: Mongo::WriteConcern.new(w: "majority")
)

# There are two ways to perform transactions:

collection.with_session(default_transaction_options: transaction_options) do |collection, session|
  puts collection.find.to_a.to_json # => "[]"

  # 1. by calling the `with_transaction` method.

  # `with_transaction` will commit after the block ends.
  # if the block raises, the transaction will be aborted.
  session.with_transaction do
    collection.insert_one({_id: 1})
    collection.insert_one({_id: 2})
  end
  puts collection.find.to_a.to_json # => [{"_id":1},{"_id":2}]

  # The transaction below will be aborted because the block raises an Exception.
  begin
    session.with_transaction do
      collection.insert_one({_id: 3})
      raise "Interrupted!"
      collection.insert_one({_id: 4})
    end
  rescue e
    puts e # => Interrupted!
  end
  puts collection.find.to_a.to_json # => [{"_id":1},{"_id":2}]

  # 2. by calling the `start_transaction`, `commit_transaction` and `abort_transaction` methods.
  session.start_transaction
  collection.insert_one({_id: 3})
  # The transaction is isolated, reading outside of the session scope does not return documents impacted by the transaction…
  puts database["collection"].find.to_a.to_json # => [{"_id":1},{"_id":2}]
  # but reading within the session scope does.
  puts collection.find.to_a.to_json # => [{"_id":1},{"_id":2},{"_id":3}]
  session.commit_transaction
  # The transaction is now committed and visible outside of the transaction scope.
  puts collection.find.to_a.to_json             # => [{"_id":1},{"_id":2},{"_id":3}]
  puts database["collection"].find.to_a.to_json # => [{"_id":1},{"_id":2},{"_id":3}]
end

Links

Specifications

The goal is to to be compliant with most of the official MongoDB set of specifications.

✅ Implemented

The following specifications are implemented:

⏳Next

The following specifications are to be implemented next:

Contributing

Fork it (https://github.com/elbywan/cryomongo/fork)
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create a new Pull Request

Contributors

elbywan - creator and maintainer
paulocoghi - contributor

Credit

Icon made by Smashicons from www.flaticon.com.

Name		Name	Last commit message	Last commit date
Latest commit History 164 Commits
.github		.github
docs		docs
spec		spec
src		src
.crystalline_main.cr		.crystalline_main.cr
.editorconfig		.editorconfig
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
ROADMAP.md		ROADMAP.md
icon.svg		icon.svg
shard.yml		shard.yml

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

cryomongo

A MongoDB driver written in pure Crystal.

Roadmap & Checklist

Cryomongo is a high-performance MongoDB driver written in pure Crystal. (i.e. no C dependencies needed.)

Installation

Usage

Minimal working example

Complex example with serialization

Features

Conventions

Documentation

Connection

Authentication

Basic operations

Bulk operations

Indexes

GridFS

Change streams

Raw commands

Concerns and Preference

Commands Monitoring

Causal Consistency

Transactions

Specifications

Contributing

Contributors

Credit

About

Uh oh!

Releases 5

Packages

Uh oh!

Contributors

Uh oh!

Languages

Uh oh!

Folders and files

Latest commit

History

Repository files navigation

cryomongo

A MongoDB driver written in pure Crystal.

Roadmap & Checklist

Cryomongo is a high-performance MongoDB driver written in pure Crystal. (i.e. no C dependencies needed.)

Installation

Usage

Minimal working example

Complex example with serialization

Features

Conventions

Documentation

Connection

Authentication

Basic operations

Bulk operations

Indexes

GridFS

Change streams

Raw commands

Concerns and Preference

Commands Monitoring

Causal Consistency

Transactions

Specifications

Contributing

Contributors

Credit

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 5

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages