Bulk data api issues

Hi All

Trying to understand issues i am having with the bulk data api.

I have got the api connector and set it up to allow me to get data from within the database and then create new records on mass.

Data wise my things are setup below

Category thing (50)
pos item (3000) - a pos item is linked to the category
Allocation - loop through the category and based on user selection it copies the pos item(s) to the allocations.

The API looked like it was working fine and creating records, upon further investigation it isn’t. Out of the 50 categories lets say the user selected 40 of them. This then set the ball rolling and category 1 to 10 all had 30 different pos items associated to each.

The API would create 30 for some of them, but then for others it might be 29, might be 27. Basically omitting some of them for no reason at all. I ran the api again against the same data source and it did it again, however some of the ones it missed previously were now correct and others were wrong.

Has anyone come across similar issues and if so how did you resolve? my only assumption is that the backend workflow is not completing in full before the next loop. With the new WU capacity i don’t want to go down creating thousands of lines one by one, not do i want to have to run additional workflows after the event to check everything has imported correctly.

Thanks