Processing Background Jobs Using Sidekiq Gem in Rails 7 (Part-I)

Photo by Lala Azizli on Unsplash

As a Rails developer, writing code that executes in the background asynchronously is a very common task that every backend developer performs almost all the time.

For example, to send a welcome email to a new customer or to traverse thousands of records for complex computations, background processing is used.

Luckily, we’ve tools available that save tons of our time by taking care of all the heavy lifting and providing a simple interface. Let’s look at one of them, Sidekiq!

What is Sidekiq?

Sidekiq is a popular background job processing library for Ruby on Rails applications. It provides a simple and efficient way to offload time-consuming tasks, as mentioned above, from the main request-response cycle of a Rails application.

There are three main components that Sidekiq relies on:

Redis

Sidekiq uses Redis, an in memory data store, to store jobs that are processed by the worker.

Client

It is the Ruby or Rails process that creates jobs that are processed in the background.

Server

This is responsible for running jobs by pulling them from the Redis queue.

Too much text? let’s build something now!

Prerequisites

As mentioned, Sidekiq relies on Redis to enqueue background jobs. So, make sure you have Redis server running locally.

Step 1

Create a new project or skip this step if you have an already set up project.

$ rails new sidekiq_tutorial

Step 2

Inside this new project write:

$ bundle add sidekiq

Step 3

Now, we are ready to create new jobs and push them to Sidekiq worker. Let’s generate a new job:

$ rails generate sidekiq:job my_first_job

This will generate app/sidekiq/my_first_job.rb and a test file. Let’s edit the contents of the job file to look like this:

class MyFirstJob
  include Sidekiq::Job

  def perform(name,age)
    puts "I am #{name}, running my first job at #{age}"
    #any other valid Ruby/Rails code goes here!
  end
end

Here, we are only logging some text onto the console, but you can run almost any valid code here.

Before we run this job, we need to make sure that Redis is configured correctly.

Redis Configurations

There are two ways you can do this, one is by using REDIS_URL environment variable( by using dotenv-rails gem) or if you need more complex setup, you can create a sidekiq.rb file in the config/initializers.

Using .env file

REDIS_URL=redis://redis.example.com:7372/0

Using config/initializers/sidekiq.rb

Sidekiq.configure_server do |config|
  config.redis = { url: 'redis://redis.example.com:7372/0' }
end

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://redis.example.com:7372/0' }
end

Note: You need either of these configuration to connect with Redis.

If you are good at Docker, you can use a simple Redis container to run Redis and connect your Rails app with it.

Queuing the Jobs

Let’s jump into Rails’ console and run the job:

$ rails c

3.0.0 :001 > MyFirstJob.perform_async "Affan", 31
=> "5a4c435ddd2295a6104c8fcb"

As you can see the job has been successfully created.

Running the Jobs

Now that the job is in the queue, we need to run the Sidekiq worker to execute the job. Open another terminal window and execute:

$ bundle exec sidekiq

And you’ll notice something like this if everything works fine.

Once you finish running jobs, you can terminate the worker using ctrl+c.

Conclusion

In the first part of this tutorial, we discussed the Sidekiq, how it processes the background jobs and created a simple job in a new Rails project. We also discussed some of the configuration options to connect Sidekiq with your Rails app. We’ll discuss some of the advanced things related to this topic in part-2 of the story :)