This guide will walk you through an example MVC web application rendered with razor pages. It is intended to show how you can integrate a Runly job into a web application workflow. In this example, after a user clicks an action button, a long-running, asynchronous job will be queued while providing real-time feedback to the user.

Overview

This example provides a button that when clicked will simulate sending email to a large number of users.

When implementing something like this, the easiest thing you can do is just send the email inline in the controller action. However, doing it like this can lead to a poor user experience. Waiting for all the email to be sent could take seconds to minutes depending on how many messages are being sent and the latency of your email server.

If the user has to wait for that operation to complete before receiving a response from the server, they might think your app is broken and, worse, they might refresh the page and/or try the operation again potentially leaving your data in an inconsistent state if you don’t explicitly handle this case.

Things like this are easy to make it past testing since the naive implementation will work just fine with a small dataset. It is only when real-world usage and/or when things start to scale to large numbers of users/records that things start to fall apart.

Runly aims to help developers fall into the pit of success by making it easy to separate the long-running, non-critical work out from your web application. It also gives you fault tolerance and retry mechanisms built-in.

In this example:

  1. the user clicks the action button
  2. a long-running job to “send” emails is queued
  3. the web request returns immediately
  4. the user is provided real-time feedback on the state of their action

This makes the user happy since they get a nice experience despite throwing a large amount of data at your application. It makes developers happy since they can implement a scalable solution with a small level of effort. It makes managers happy since developers are more productive.

And really, don’t we all deserve to be happy? 😉

Setup

Clone the examples repository and switch to the web-app directory:

git clone https://github.com/runlyio/core-dotnet.git
cd core-dotnet/examples

The example solution is broken up into two projects:

  • a Web project which is the MVC web application
  • a Processes project which contains the Runly jobs

Pack the Processes project and upload the resulting nupkg to your Runly account.

cd Processes
dotnet pack -c Release

Configure Your Runly Infrastructure

The example web application will use the Runly API to queue a job on a specific environment when a user performs an action. It will also show the status and a progress bar of the long-running job on the frontend. In order to do that, you will need to generate application-specific API keys to authenticate the application to the Runly API.

Create two applications:

  • Example Server Application: Create this as a privileged (non-public) app since the intention is to keep this key secret on the server.
  • Example Client Application: Create this as a public app since we will use this one on the frontend in a public environment (the browser).

Learn more about the difference between public and privileged apps.

Once you have generated your application API keys, copy them along with your organization ID and environment name (where you would like jobs queued) to the runly section of the appsettings.json file:

{
	"runly": {
		"org": "myorg",
		"env": "production",
		"secretKey": "my-server-application-api-key",
		"publishableKey": "my-js-client-application-api-key"
	}
}

Your organization ID can be found from your dashboard by navigating to Settings ➡️ General ➡️ API ID.

Service Registration

In Startup.cs, we register the Runly client services using values from the appsettings.json file:

services.AddRunlyApi(s =>
{
	var opts = s.GetRequiredService<IOptionsSnapshot<RunlyOptions>>().Value;
	return opts.ApiKey;
});

This will allow us to queue jobs on our Runly infrastructure from our controllers. Learn more about how to integrate the Runly.Client with your code.

Job Queue Abstraction

Notice we also register a ProcessQueue abstraction:

services.AddTransient<IProcessQueue, RunlyProcessQueue>();

This is not required to queue jobs on Runly but it can be helpful to consolidate the Runly-related code to a single class. This allows you to pull configuration values such as organization, environment, or package settings so you don’t have to repeat it everywhere where you are queueing jobs in your code.

public interface IProcessQueue
{
	// provide a domain-specific method to queue processes
	Task<Guid> SendPendingInvitations();
}

public class RunlyProcessQueue : IProcessQueue
{
	readonly RunlyOptions opts;
	readonly IRunClient runs;

	public RunlyProcessQueue(
		IOptionsSnapshot<RunlyOptions> opts,
		IRunClient runs
	)
	{
		this.opts = opts.Value;
		this.runs = runs;
	}

	public async Task<Guid> SendPendingInvitations()
	{
		// this method just abstracts the specifics of queueing a job away from the app
		var run = await runs.Enqueue<InvitationEmailer, InvitationEmailerConfig>(
			opts.Org,
			opts.Env,
			new InvitationEmailerConfig
			{
				ConnectionString = opts.ConnectionString
			}
		);

		return run.Id;
	}
}

Queueing a Job

The frontend of the app is a Razor page with a single-field form to invite fictional users to your fictional app:

<form method="POST">
	<div>
		<label for="emails">Emails to Invite</label>
		<textarea id="emails" name="emails"></textarea>
	</div>

	<button type="submit">Send Invites</button>
</form>

Once a user submits this form, it will POST to the InviteController:

[HttpPost]
public async Task<IActionResult> InviteUsers(InvitationModel data)
{
	// Do the critical path work: actually get
	// the users/emails into the system.
	await InsertUsersIntoDatabase(data.EmailList);

	// Then queue the non-critical path work
	// in a separate async-job: email
	// invitations will be sent to each user.
	// This also makes the process of sending
	// emails fault tolerant in case of
	// temporary errors while sending a message.
	await processes.SendPendingInvitations();

	return View();
}

Runly shines in breaking ancillary concerns out of your critical path. In this case, the act of getting the users into the system is the critical path. The sending of invitation emails is something that can intermittently fail, could possibly take a long time, and is something that could be retried later if it fails the first time. Breaking out this functionality into a Runly job makes the web method more robust, easier to reason about, and more fault tolerant.

Running the Job

It doesn’t take much to make the job fault tolerant. When we inserted the invited users into our contrived system, we added a flag, HasBeenEmailed, that indicates whether we sent them their invitation email yet. When our InvitationEmailer job starts, it will only process users who have not yet received their invitation email:

public override IAsyncEnumerable<string> GetItemsAsync()
{
	return db.QueryAsync<string>("select [Email] from [User] where HasBeenEmailed = 0").ToAsyncEnumerable();
}

Each pending email is returned as a separate item to process. This means the job can run tasks in parallel to send multiple emails at a time.

public override async Task<Result> ProcessAsync(string email, DbConnection db, IEmailService emails)
{
	// send our fake email
	await emails.SendEmail(email, "You are invited!", "Join us. We have cake.");

	// mark the user as invited in the database
	await db.ExecuteAsync("update [User] set HasBeenEmailed = 1 where [Email] = @email", new { email });

	return Result.Success();
}

The ProcessAsync method takes the non-thread-safe dependencies db and emails as parameters so that it will get a new instance of those dependencies per parallel task. Learn more about dependency injection in jobs.

The meat of the method sends the email and then marks the email as having been sent in the database. If any single item fails, the user won’t have their email flag flipped in the database so that it can be retried later on subsequent job runs.

Read more about how Runly jobs provide fault-tolerance out of the box.

Providing Feedback to the User

This example makes use of runly.js in Results.cshtml to provide results of the long-running job to the user:

@inject IOptionsSnapshot<RunlyOptions> opts
@model RunResultsModel

@section Scripts {
	<script type="text/javascript" src="https://cdn.runly.io/v1.0.0/runly.js"></script>
}

<div
	data-runly-token="@opts.Value.PublishableKey"
	data-runly-org="@opts.Value.Org"
	data-runly-run="@Model.RunId">Loading run results...</div>

It uses the injected RunlyOptions along with a RunResultsModel from the InviteController to build a div that will show run results. This will render an animated progress bar as emails are sent.

progress bar

There are many options to integrate your frontend with Runly that range from minimal effort to maximum customization. Learn more about how to integrate your frontend.

Running the Application

To start the web application on localhost:5000:

Via Visual Studio

  1. Open the WebApp.sln in Visual Studio.
  2. Set the Web project as the Startup Project.
  3. Hit F5 to start debugging the web app.

Via Visual Studio Code

  1. Open the web-app folder in VS Code.
  2. Hit F5 to start debugging the web app.

Via .NET Core CLI

cd to the Web folder:

cd web-app/Web
dotnet run

Database

The example web application saves data to a local SQLite database in example.db. The web app creates this file on startup if it does not exist. If you are curious, you can use a tool like DB Browser for SQLite to explore the data in the example.db file.

The job and the web app both need access to the same database, the file on your file system. In order for the job to run correctly for this example, you will need to run your organization’s Runly node on the same machine that you run your webapp.

In the real world, you would probably have access to a “real” database server. In that case, your job and web app can run on different machines as long as both have access to the database server.

Retrying Job Runs

The example InvitationEmailer job is set to fail with a fake Internet Down error for every 100th email that it sends. This gives you an opportunity to try out the retry functionality built in to the runly.js components.

retry job run

On second run, the InvitationEmailer will only process the 10 failed items and succeed. You can hide this functionality from your users if you like and instead rely on setting up the InvitationEmailer to run on a schedule to pick up any email messages it missed. Explore more ways to make your jobs fault tolerant.

Troubleshooting

401 Unauthorized When Submitting Form

After submitting the sample form, you receive an HttpRequestException. Make sure to create two application keys and update your appsettings.json file.

404 Not Found When Submitting Form

After submitting the sample form, you receive an HttpRequestException:

{
	"message": "Organization does not exist",
	"helpLink": null
}

Ensure you’ve updated appsettings.json with your organization ID. The ID is different from the organization name. You can find the ID by clicking on Settings from your organization’s dashboard and then copying the API ID.

Next Steps