7848 Add RQ API

[arthanson]

Fixes: #7848

Here is a testing command to add some tasks for testing the API response, create a temporary create_task management command within core.

Used APIView on the API's and not ViewSet as they don't fit with a ViewSet / only one operation per API can't Get, List, Delete, etc. and the URL structure doesn't directly conform with the pk used by ViewSet, however the side-effect of this is it doesn't appear in the browsable API.

Note: some of the status on these tasks might not be correct as they haven't been correctly moved to the queues by RQ, we are just directly placing them there.

from django_rq.management.commands.rqworker import Command as _Command

from datetime import datetime, timedelta
from django_rq import get_queue
from rq.registry import DeferredJobRegistry, FailedJobRegistry, FinishedJobRegistry, ScheduledJobRegistry, StartedJobRegistry


class Command(_Command):
    # Dummy worker functions
    @staticmethod
    def dummy_job_default():
        return "Job finished"

    @staticmethod
    def dummy_job_high():
        return "Job finished"

    @staticmethod
    def dummy_job_failing():
        raise Exception("Job failed")

    def handle(self, *args, **options):
        queue = get_queue('default')

        job = queue.enqueue(self.dummy_job_default)
        registry = FinishedJobRegistry(queue.name, queue.connection)
        registry.add(job, -1)

        job = queue.enqueue(self.dummy_job_default)
        registry = FailedJobRegistry(queue.name, queue.connection)
        registry.add(job, -1)

        job = queue.enqueue(self.dummy_job_default)
        registry = DeferredJobRegistry(queue.name, queue.connection)
        registry.add(job, -1)

        job = queue.enqueue(self.dummy_job_default)
        registry = StartedJobRegistry(queue.name, queue.connection)
        registry.add(job, -1)

        job = queue.enqueue(self.dummy_job_default)
        when = datetime.now() + timedelta(hours=8)
        registry = ScheduledJobRegistry(queue.name, queue.connection)
        registry.schedule(job, when)

[bctiemann]

Should these be POST operations maybe?

[jeremystretch]

Yes; any operation which alters data must use a POST, PATCH, PUT, or DELETE request.

[jeremystretch]

It seems like we could combine all these task lists into a single view and pass the status (deferred, failed, etc.) as a URL path variable.

[jeremystretch]

Yes; any operation which alters data must use a POST, PATCH, PUT, or DELETE request.

[jeremystretch]

All the individual task views can probably be combined into a single TaskView class with a custom action for each operation. This would eliminate a lot of boilerplate and simplify the URL paths.

[jeremystretch]

None of these paths appear in the browseable API, though I'm not sure why.

[jeremystretch]

These list views should replicate the existing API convention, which enables pagination of the results:

{
    "count": 24,
    "next": null,
    "previous": null,
    "results": [...]
}

[jeremystretch]

Suggested change

	jobs = get_rq_jobs_from_status(queue, status)
	return get_rq_jobs_from_status(queue, status)

[jeremystretch]

Please add docstrings for these functions.

[jeremystretch]

We might want to move the 404 handling out to the UI & API views, to ensure that these functions can be used cleanly internally e.g. within tests.