Think this will work:

* item 1
 ** item a +
    paragraph 1 inside item a
+
--
*** item a.1 inside paragraph 1 inside item a +
    paragraph a.1 inside item a.1
*** item a.2 inside paragraph 1 inside item a +
    paragraph a.2 inside item a.1
--
+
paragraph 2 inside item a
 ** item b +
    paragraph 1 inside item b
* item 2

Tested with Asciidoctor 1.5.6.1.

From the documentation: http://asciidoctor.org/docs/user-manual/#cell

The direct Asciidoctor PDF rendering do not support this yet. See #6

This is what I get:

With paragraph:

[cols="1"]
|===

a|First paragraph

second paragraph

|===

The relevant thing is the empty line between |== and a|your cell

With the HTML renderer:

With the PDF renderer:

With list:

It works the same way:

[cols="1"]
|===

a|First paragraph

* second paragraph

|===

With the HTML renderer:

With the PDF renderer:

A possible solution might be to use the DocBook Pipeline with jDocBook as in this example docbook-pipeline-jdocbook-example. With this setup I get the expected output: