GET API call not fetching all data

[diegoromanoff2]

Hi all!

I’m encountering a very strange issue with API calls.

I call a scraping API that returns an array of objects and save them in the DB through this process:
a) i set-up a backend API workflow that creates a new thing in the DB
b) in the frontend workflow I 1) call the API and 2)“Schedule the API workflow on a list” from point a).

The strange thing is the process is working BUT it only fetches a few results in the DB. For ex. I know for a certain call the array has >40 objects but Bubble only saves 5.

Anyone encountered this issue?

Thanks a lot and have a nice day
D.

[Jici]

A) Are you on new pricing WU plans?
B) Did you inspect step by step to see the list of item sent
C) Can you share screenshot of your schedule on a list action
D) Can you share the API payload

[diegoromanoff2]

Hi @Jici thanks for the reply.

Here’s the asked info:
A) i’m on the free trial of the starter plan

B) I tried to inspect the workfow in debugger and it sends the few (< than optimal) objects it retrieves (e.g. I expect 40 objects, it sends 9)

C)

D) API payload for one object (the API retrieves job postings):

{
  "status": "OK",
  "request_id": "164fd697-4646-46ff-a9e8-c5b80f866ed5",
  "parameters": {
    "query": "python developer in texas, usa",
    "page": 1,
    "num_pages": 1
  },
  "data": [
    {
      "job_id": "37QmCYhsRONU67ceAAAAAA==",
      "employer_name": "Collabera",
      "employer_logo": "https://collabera.com/wp-content/uploads/2022/06/Mask-group-2.svg",
      "employer_website": "http://www.collabera.com",
      "employer_company_type": "Computer Services",
      "job_publisher": "LinkedIn",
      "job_employment_type": "CONTRACTOR",
      "job_title": "Python Developer",
      "job_apply_link": "https://www.linkedin.com/jobs/view/python-developer-at-collabera-3906621384",
      "job_apply_is_direct": false,
      "job_apply_quality_score": 0.6105,
      "apply_options": [
        {
          "publisher": "LinkedIn",
          "apply_link": "https://www.linkedin.com/jobs/view/python-developer-at-collabera-3906621384",
          "is_direct": false
        }
      ],
      "job_description": "Must haves:\n• Previous Application Programming/Development experience designing, developing and deploying high-volume, multi-tiered, distributed transactional applications.\n• Deep knowledge of Python Django Framework\n• Deep knowledge of Angular, ES6/HTML5/CSS3\n• Strong knowledge of MySQL databases\n• Deep understanding of Git version control system\n• Strong understanding of unit testing frameworks\n• Experience working in Agile development process and good understanding of various phases of Software Development Lifecycle\n• Experience in data access technologies and patterns\n• Relational database experience and proven abilities in SQL and data modeling is required. Must be aware of SQL Objects, query optimization, and join conditions.\n• Experience with component based web application frameworks\n• Conceptual understanding of Object Oriented Programming and SOLID design principles.\n• Ability to communicate, written and verbal, with technical and non-technical cross-functional teams\n• .NET Core , Jenkins, MSBuild, Ansible, XLR, Automated testing",
      "job_is_remote": false,
      "job_posted_at_timestamp": 1713789734,
      "job_posted_at_datetime_utc": "2024-04-22T12:42:14.000Z",
      "job_city": "Plano",
      "job_state": "TX",
      "job_country": "US",
      "job_latitude": 33.019844,
      "job_longitude": -96.69888,
      "job_benefits": null,
      "job_google_link": "https://www.google.com/search?gl=us&hl=en&rciv=jb&q=python+developer+in+texas,+usa&start=0&ibp=htl;jobs#fpstate=tldetail&htivrt=jobs&htiq=python+developer+in+texas,+usa&htidocid=37QmCYhsRONU67ceAAAAAA%3D%3D",
      "job_offer_expiration_datetime_utc": "2024-05-22T12:41:39.000Z",
      "job_offer_expiration_timestamp": 1716381699,
      "job_required_experience": {
        "no_experience_required": false,
        "required_experience_in_months": null,
        "experience_mentioned": true,
        "experience_preferred": false
      },
      "job_required_skills": null,
      "job_required_education": {
        "postgraduate_degree": false,
        "professional_certification": false,
        "high_school": false,
        "associates_degree": false,
        "bachelors_degree": true,
        "degree_mentioned": false,
        "degree_preferred": false,
        "professional_certification_mentioned": false
      },
      "job_experience_in_place_of_education": false,
      "job_min_salary": null,
      "job_max_salary": null,
      "job_salary_currency": null,
      "job_salary_period": null,
      "job_highlights": {
        "Qualifications": [
          "Previous Application Programming/Development experience designing, developing and deploying high-volume, multi-tiered, distributed transactional applications",
          "Deep knowledge of Python Django Framework",
          "Deep knowledge of Angular, ES6/HTML5/CSS3",
          "Strong knowledge of MySQL databases",
          "Deep understanding of Git version control system",
          "Strong understanding of unit testing frameworks",
          "Experience working in Agile development process and good understanding of various phases of Software Development Lifecycle",
          "Experience in data access technologies and patterns",
          "Relational database experience and proven abilities in SQL and data modeling is required",
          "Must be aware of SQL Objects, query optimization, and join conditions",
          "Experience with component based web application frameworks",
          "Conceptual understanding of Object Oriented Programming and SOLID design principles",
          "Ability to communicate, written and verbal, with technical and non-technical cross-functional teams",
          ".NET Core , Jenkins, MSBuild, Ansible, XLR, Automated testing"
        ]
      },
      "job_job_title": null,
      "job_posting_language": "en",
      "job_onet_soc": "15113200",
      "job_onet_job_zone": "4",
      "job_occupational_categories": null,
      "job_naics_code": "541511",
      "job_naics_name": "Custom Computer Programming Services"
    }
 ]
}

Let me know if you need more infos

thanks again!

[Jici]

Your settings seem fine. Try to add interval, but with the new release for Schedule on a list, it shouldn’t be a problem.
If the Result of step 1 return 40 items, I don’t see any issue. You should reach Bubble support

[diegoromanoff2]

I did some diving and maybe there’s some problems with the API params, I’ll try something else and then in case reach to the support.

Thanks anyways!