I believe I found the answer! Startup.cs is not run before the first call to the API is made. After calling a simple Test method in one of my controllers, the BackgroundProcessing method was called That's a bit annoying, as I was hoping I later could create a backgroundworker that loads a lot of data into memory instead of it happening when the first call is made

These background tasks will run for the entirety of the applications life, and will be reading azure queues so spending a lot of time sitting doing nothing.

I agree with others' comments: by far the best solution is to move these out of the ASP.NET process completely. Azure Functions, for example, has built-in support for Azure Storage Queues. Benefits of Azure Functions over ASP.NET in-process include:

• Your API (ASP.NET) can scale independently from your backend (Azure Function).
• Scaling on the backend is automatic. Up as high as you want, potentially down to zero.
• The backend process has its own thread pool, so there's no interference with the ASP.NET thread pool.

I have read that for I/O you shouldn't use Task.Run() as it's inefficient

Task.Run should be avoided in ASP.NET's request processing pipeline. It's always inefficient regardless of the type of work being done.

Task.Run can be useful for background tasks in an ASP.NET process, in particular if there's blocking work done during startup of the background task.

The main thing with Task.Run on ASP.NET is that you don't want to interfere with the ASP.NET thread pool heuristics. Regularly queueing work to the thread pool is a problem, since you're stealing threads and then re-injecting them on a regular basis. So using Task.Run when processing each HTTP request or when processing each queue item would be a bad idea. A single Task.Run done once at startup doesn't matter.

It seems if I don't use Task.Run() then every time my background task gets something from the queue and so the code after await continues, then it will be blocking some main thread in ASP that spooled up the task in the first place.

This is a valid concern if there's blocking work being done during your message processing.

One way to think about async/await is that each method is broken up into pieces (at each await point), and each of those pieces is scheduled separately, one at a time. Each time a piece is scheduled, it takes a thread pool thread, runs the piece, and then returns the thread. As long as the work is fast, this is fine, but if one of those pieces has blocking work, then the ASP.NET thread pool heuristics can be impacted.

I asked myself a similar question and made some search but couldn't find a good answer. I solved the issue running every background service in Task.Run with a cancellation token from BackgroundService.ExecuteAsync() I have 2 services like you.

public class BackgroundService1: BackgroundService
{
    public BackgroundService1()
    {
    }
    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        Task.Run(async () =>
        {
            await DoWork(stoppingToken);
        }, stoppingToken);
        return Task.CompletedTask;
    }
}

//Second service is just like the first one:

public class BackgroundService2: BackgroundService
{
    public BackgroundService2()
    {
    }
    protected override Task ExecuteAsync(CancellationToken stoppingToken)
    {
        Task.Run(async () =>
        {
            await DoWork(stoppingToken);
        }, stoppingToken);
        return Task.CompletedTask;
    }
}

and register them in Program.cs

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostContext, services) =>
            {
                services.AddHostedService<BackgroundService1>();
                services.AddHostedService<BackgroundService2>();
            })
            .UseWindowsService()

You need to look at a "queued background service" where you can submit "jobs" to it and it will perform those jobs in a background queue.

The work flow goes like this:

1. Caller sends a request to the service with some parameters
2. Service generates a "job" object and returns an ID immediately via 202 (accepted) response
3. Service places this job in to a queue that is being maintained by a BackgroundService
4. Caller can query the job status and get information about how much has been done and how much is left to go using this job ID
5. Service finishes the job, puts the job in to a "completed" state and goes back to waiting on the queue to produce more jobs

Here is a very long-winded explanation on how it works: https://stackoverflow.com/a/63429262/1204153

Here is an example I made a while back: https://github.com/sonicmouse/ComputationService

Updated 03-2022, read it on the bottom!

Updated 04-2020, read it on the bottom!

@Panagiotis Kanavos gave an answer in the comments of my question but it did not post it as an actual answer; this answer is dedicated to him/her.

I used a Timed background service like the one from Microsoft docs to create the service.

internal class TimedHostedService : IHostedService, IDisposable
{
    private readonly ILogger _logger;
    private Timer _timer;

    public TimedHostedService(ILogger<TimedHostedService> logger)
    {
        _logger = logger;
    }

    public Task StartAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Timed Background Service is starting.");

        _timer = new Timer(DoWork, null, TimeSpan.Zero, 
            TimeSpan.FromSeconds(5));

        return Task.CompletedTask;
    }

    private void DoWork(object state)
    {
        _logger.LogInformation("Timed Background Service is working.");
    }

    public Task StopAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Timed Background Service is stopping.");

        _timer?.Change(Timeout.Infinite, 0);

        return Task.CompletedTask;
    }

    public void Dispose()
    {
        _timer?.Dispose();
    }
}

In my case I made the _timer call async by doing new Timer(async () => await DoWorkAsync(), ...).

In the future, an extension could be written that makes a class like this available in the Extensions repo because I think this is quite useful. I posted the github issue link in the description.

A tip, if you plan on reusing this class for multiple hosted services, consider creating a base class that contains the timer and an abstract PerformWork() or something so the "time" logic is only in one place.

Thank you for your answers! I hope this helps someone in the future.

Update 04-2020:

Injecting a scoped service in here is not possible with the normal Core service collection DI container, out of the box. I was using autofac which made it possible to use scoped services like IClassRepository in the constructor because of wrong registration, but when I started working on a different project that used only AddScoped<>(), AddSingleton<>(), AddTransient<>() we figured out that injecting scoped things do not work because you are not in a scoped context.

In order to use your scoped services, inject a IServiceScopeFactory (Easier to test with) and use CreateScope() which allows you to use scope.GetService() with a using statement :)

Update 03-2022: This post has gotten LOTS of views and attention, but I have to say I am no longer a big fan of my solution. I would propose different solutions:

• Use hangfire or quartz instead if you want the code to just run in backgroundservice
• take a look at kubernetes cronjobs if you run in a kubernetes environment
  • This has the benefit of only running your code when required, saving resources compared to running a project 24/7 and only executing a job every day at 3 AM, for example
• take a look at Azure Functions/AWS Lambda on a timer
  • this is probably cheaper and easier to maintain than making your own timed hosted services. It might be more difficult to integrate into a k8s environment, though.

The downsides of the solution posted in this answer are:

• You need to manage a lot of things yourself that the other options do for free. For example:
  • What if your app was down when it should have ran the job?
  • What if your job takes too long and another one starts?
  • Logging and monitoring
• I am still unsure about the async support in this solution. I never really figured out if this solution is "correct"
• I also do not like that DI is not supported out of the box. Quartz.Net does support this.
• It isn't flexible compared to quartz.