Configuration

Building and running a job starts with configuration. Every job requires a config, which can be the Config class by itself or a class that extends it. When writing a job, you’ll need to specify the type of config to use when extending the Job class in the TConfig generic type parameter.

public class CopyDirectory : Job<CopyDirectoryConfig, string>

Extending Config allows a job to add properties that it requires to run. In this example, the CopyDirectory job needs to know which directory to copy and where to copy it to. These properties can be used in any method of the job, being accessed via the Config property of the job.

public class CopyDirectoryConfig : Config
{
	public string Source { get; set; }
	public string Destination { get; set; }
}

Config Class

All configs extend the Config class, which has properties necessary to run the job. The base config is grouped into three “sections” using classes to group related properties. This sort of structure is useful for managing lots of configuration parameters.

public class Config
{
	public JobConfig Job { get; }

	public ExecutionConfig Execution { get; }

	public RunlyApiConfig RunlyApi { get; }
}

The Job section identifies the package, version and type of job. This information tells the node how to start the package and hand off the config to the job.

The Execution section determines how threads are manged, how failures are handled, and where the results are persisted.

The RunlyApi section tells the node and job how to communicate with the Runly API.

Extending Config

Extending the Config is simple, just add properties as needed. As configs grow in size and complexity, there are a couple of strategies to stay organized.

As is the case with the base Config class, when you find yourself with lots of properties in a config, they could be grouped and moved to separate classes that are then added as properties on the config. Configs can also be extended multiple times to group properties for more than one job in a base config class.

Dependencies That Need Config

Some dependencies also require configuration, such as a database requiring a connection string. This type of configuration property may be necessary to have on a large number of configs. Creating an interface for this dependency’s config can allow it to be completely uncoupled from the job config.

public interface IDatabaseConfig
{
	string ConnectionString { get; }
}

public class CensusConfig : Config, IDatabaseConfig
{
	public string BaseUrl { get; set; }

	public string ConnectionString { get; set; }
}

Rather than getting a specific job’s config to add the dependency, the setup can now look for the interface.

public static IServiceCollection AddCensusJobs(this IServiceCollection services)
{
	services.AddScoped<IDatabase>(s => new Database(
		s.GetRequiredService<IDatabaseConfig>().ConnectionString
	));

	return services;
}

Serialization

All properties on a config class must be serializable. Configs are serialized to JSON, which can then be persisted or transferred. This is what is used to queue a job in the Dashboard and it is what the Job CLI reads out of a file when a job is run.

Default Configs

Every job has a default config, which is a newly instantiated instance of the config that is then serialized without any properties being modified, capturing the default state of the config. This is used as the template when selecting a job to queue in the Dashboard.

Deserialization and Filling In the Blanks

When a JSON config is deserialized back into a config class, any properties that are missing from the JSON will keep the value in the default config, which may be null, 0, an empty string, or another value depending on how the config is initialized. This is convenient because a JSON config can be extremely minimal by just containing the properties that it needs to provide a value for and ignoring those that don’t matter.

To make configs even easier to deal with, the Runly API allows you to reduce the job section to a single string with the job type. This does not deserialize directly to a config, instead the API looks up the current version of the package containing the type and rewrites the job section to be deserializable. The following JSON is a valid config for the CopyDirectory job.