Injected Hosted services start with the app and stops with it. Framework doesn't provide you manual control. There should be underlying reasons for that like. Of course, you can implement your own solutions with threads but if your wish to go on with Microsoft's built-in solutions you can use Queued Background Task. You will start a background service with an empty task queue. Service will be listening to your first task to be added to the queue. You can add a task whenever you wish then it will be running at background.

You still can use IHostedService as base for background tasks in combination with BlockingCollection.

Create a wrapper for BlockingCollection so we can inject it as singleton.

public class TasksToRun
{
    private readonly BlockingCollection<SingleTaskData> _tasks;

    public TasksToRun() => _tasks = new BlockingCollection<SingleTaskData>(new ConcurrentQueue<SingleTaskData>());

    public void Enqueue(SingleTaskData taskData) => _tasks.Add(settings);

    public TaskSettings Dequeue(CancellationToken token) => _tasks.Take(token);
}

For background process we can use "built-in" implementation of IHostedService - Microsoft.Extensions.Hosting.BackgroundService.

public class TaskProcessor : BackgroundService
{
    private readonly TasksToRun _tasks;

    public TaskProcessor(TasksToRun tasks) => _tasks = tasks;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await Task.Yield(); // This will prevent background service from blocking start up of application

        while (cancellationToken.IsCancellationRequested == false)
        {
            try
            {
                var taskToRun = _tasks.Dequeue(_tokenSource.Token);


                await ExecuteTask(taskToRun);               
            }
            catch (OperationCanceledException)
            {
                // execution cancelled
            }
            catch (Exception e)
            {
                // Catch and log all exceptions,
                // So we can continue processing other tasks
            }
        }
    }
}

Then we can add new tasks from the controller without waiting for them to complete

public class JobController : Controller
{
    private readonly TasksToRun _tasks;

    public JobController(TasksToRun tasks) => _tasks = tasks;

    public IActionResult PostJob()
    {
        var taskData = CreateSingleTaskData();

        _tasks.Enqueue(taskData);

        return Ok();
    }
}

Wrapper for blocking collection should be registered for dependency injection as singleton

services.AddSingleton<TasksToRun, TasksToRun>();

Register background service

services.AddHostedService<TaskProcessor>();

Updated 03-2022, read it on the bottom!

Updated 04-2020, read it on the bottom!

@Panagiotis Kanavos gave an answer in the comments of my question but it did not post it as an actual answer; this answer is dedicated to him/her.

I used a Timed background service like the one from Microsoft docs to create the service.

internal class TimedHostedService : IHostedService, IDisposable
{
    private readonly ILogger _logger;
    private Timer _timer;

    public TimedHostedService(ILogger<TimedHostedService> logger)
    {
        _logger = logger;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Timed Background Service is starting.");

        _timer = new Timer(DoWork, null, TimeSpan.Zero, 
            TimeSpan.FromSeconds(5));

        return Task.CompletedTask;
    }

    private void DoWork(object state)
    {
        _logger.LogInformation("Timed Background Service is working.");
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Timed Background Service is stopping.");

        _timer?.Change(Timeout.Infinite, 0);

        return Task.CompletedTask;
    }

    public void Dispose()
    {
        _timer?.Dispose();
    }
}

In my case I made the _timer call async by doing new Timer(async () => await DoWorkAsync(), ...).

In the future, an extension could be written that makes a class like this available in the Extensions repo because I think this is quite useful. I posted the github issue link in the description.

A tip, if you plan on reusing this class for multiple hosted services, consider creating a base class that contains the timer and an abstract PerformWork() or something so the "time" logic is only in one place.

Thank you for your answers! I hope this helps someone in the future.

Update 04-2020:

Injecting a scoped service in here is not possible with the normal Core service collection DI container, out of the box. I was using autofac which made it possible to use scoped services like IClassRepository in the constructor because of wrong registration, but when I started working on a different project that used only AddScoped<>(), AddSingleton<>(), AddTransient<>() we figured out that injecting scoped things do not work because you are not in a scoped context.

In order to use your scoped services, inject a IServiceScopeFactory (Easier to test with) and use CreateScope() which allows you to use scope.GetService() with a using statement :)

Update 03-2022: This post has gotten LOTS of views and attention, but I have to say I am no longer a big fan of my solution. I would propose different solutions:

• Use hangfire or quartz instead if you want the code to just run in backgroundservice
• take a look at kubernetes cronjobs if you run in a kubernetes environment
  • This has the benefit of only running your code when required, saving resources compared to running a project 24/7 and only executing a job every day at 3 AM, for example
• take a look at Azure Functions/AWS Lambda on a timer
  • this is probably cheaper and easier to maintain than making your own timed hosted services. It might be more difficult to integrate into a k8s environment, though.

The downsides of the solution posted in this answer are:

• You need to manage a lot of things yourself that the other options do for free. For example:
  • What if your app was down when it should have ran the job?
  • What if your job takes too long and another one starts?
  • Logging and monitoring
• I am still unsure about the async support in this solution. I never really figured out if this solution is "correct"
• I also do not like that DI is not supported out of the box. Quartz.Net does support this.
• It isn't flexible compared to quartz.

You need to look at a "queued background service" where you can submit "jobs" to it and it will perform those jobs in a background queue.

The work flow goes like this:

1. Caller sends a request to the service with some parameters
2. Service generates a "job" object and returns an ID immediately via 202 (accepted) response
3. Service places this job in to a queue that is being maintained by a BackgroundService
4. Caller can query the job status and get information about how much has been done and how much is left to go using this job ID
5. Service finishes the job, puts the job in to a "completed" state and goes back to waiting on the queue to produce more jobs

Here is a very long-winded explanation on how it works: https://stackoverflow.com/a/63429262/1204153

Here is an example I made a while back: https://github.com/sonicmouse/ComputationService

You can write main loop like this:

protected override Task ExecuteAsync(CancellationToken cancellationToken)
{
    while(!cancellationToken.IsCancellationRequested) 
    {
        try {
            await RunConsumer(cancellationToken);
        }
        catch (Exception e)
        {
            // log exception
        }
        await Task.Delay(TimeSpan.FromSeconds(30)); // To prevent restarting too often
    }
    return Task.CompletedTask;
}

The key points is

• By checking cancellationToken.IsCancellationRequested, the service will be stopped when it is requested, e.g. when the process is ending gracefully,
• Catch all exception and ignore it, so the task can be run again.
• Task.Delay ensures that the process will not be restarted too often.

In RunConsumer you can just use

await Task.Run(() => new MessageHandler().Process(stoppingToken).Start());

It is usually better to use async/await so you don't have to do the continuation and error-checking manually.