Hi,
Just released my very first plugin. You can take your users csv file and convert it to a JSON string which you can then send it to your Bubble backend workflow and upload the data. The library used to parse the csv file is called Papa Parse and its super fast (5,000 rows in 1.2 seconds).
@shawnmi6 This looks awesome! Can you share a little more info of what the workflow or required steps are after the data is parsed to get it into Bubble? Iām familiar with backend workflows in general and currently when a user uploads a csv in my apps, I pass it out to EasyCSV and then back into Bubble through a bulk data importer. Iām thinking this might be even faster and more reliable but just need a nudge in the right direction.
Also, currently when a user uploads their data, I need to tag it to a particular Team and Client. I pass the unique IDs of those two values out to EasyCSV along with the data and those UniqueIDs are put into two new columns in the csv, one for Team and one for Client. Every item in the upload gets the same value for Team and the same value for Client. Could I do something similar with this?
Tried to implement papaparse couple times without success, great work!
I think bubble limit is 1000 rows/action.
You can use the āSchedule workflow on a listā and pass the data generated as a parameter (make sure to check mark ālistā) and you can define two more parameters for user and client and pass that information to the backend workflow and then when your setting up the workflow you would have all the information you need to create a new record for each item in the list.
Thanks for the reply! Iām sure itās something simple that Iām missing, but how to I map the right column data to the right field of the data type I am creating when a CSV is uploaded?
Itās a little difficult but you would create a new endpoint in your backend workflow and then using something like Postman you will send a POST request to your endpoint while itās in āinitializingā mode and it will pick up all the fields in your CSV.
The body of the Post request needs to be the json generated by the plugin.
Ah that makes total sense.
Can I validate the csvās headers match my expected values using your plugin before scheduling the api workflow?
There isnāt a validation feature, but I assume its irrelevant because you will be hand picking each column and deciding which field to populate in your Bubble database when you are creating the workflow in your backend.
Also keep in mind the headers used in your CSV has to be exactly the same in every csv file or else your bubble backend workflow wont recognize those headers. When you run the āinitializerā it saves the names of the headers, so anything different will be ignored by bubble.
CSV File:
| First Name | | Last Name | | Age |
Shawn Lancer. 28
Bubble Database:
(field) Fname ------> First Name (CSV Field)
(field) Lname ------> Last Name (CSV Field)
(field) age ----------> Age (CSV Field)
@avern87 if you need validation and the ability to upload any csv file with any type of headers and have the user match their own headers with your pre-defined ones, then flatfile.io would be a good solution for you. I recently did a custom integration for @ksaitow1 let me know if I can help with yours if needed.
Thanks for all the help! Yea, I love FlatFileās product, but $99 a month prices me out.
Iām getting an error when I try to run this plugin on an uploaded CSV. Iām uploading, running your parse action and then attempting to set a state to the result json so I can fire off the results to detect the api parameters.
Hereās the error:
Plugin action Convert CSV to JSON error:
ReferenceError: $jscomp is not defined
at eval (eval at build_function (/var/task/index.js:55:21), :1:105)
at Object.async_fn [as async] (/var/task/index.js:242:13)
at eval (eval at build_function (/var/task/index.js:55:21), :1:75)
at /var/task/index.js:278:23
at run_fn (/var/task/u.js:594:18)
Thanks for the update. Its a known bubble bug, just fixed it. Please update the plugin and try again.
Perfect. It runs now!
Two questions for you. First, when I set the preview field to just 3 lines, the server times out. But, when I allow it to process the whole CSV, it runs without issue. Weird right?
Second, I put the json result into a text element on the page and used that to initialize the backend workflow with the right day. But, it seems detecting the data gets rid of the other parameters I put in the workflow. I want to send the CSV + a Client and a Team. Am I missing something?
@avern87 just pushed a new update added couple of new features and cleaned up some bugs.
Now you can add custom key/value pairs to your json data
Now you can have the plugin automatically send the converted data to any api endpoint
*Bubbleās servers timeout with files above 1,000 rows, so it would be a safe practice to limit the numbers of rows to 1,000.
*Preview functionality has been removed as it had a pending issue waiting to be fixed.
Would like to get your take on how (or if) your plugin would take a csv file that has row headers instead of column headers? An example is shown below where I need to take 13 columns of data (12 months and 1 description) and input that into the database. Itās almost as if I need a solution that can turn the csv on its side first.
Hi @shawnmi6
This plugin looks very interesting! Thanks for your work! Would you please share how you build the backend workflow to update the database? Also, how do you pass the JSON generated from Convert to JSON action? I would love to try this plugin but got stuck.
The plugin has the option to send off the converted data to a api endpoint. You can just crate a new backend api workflow and then take that url and paste it into the plugin endpoint field.
You might need to first initialize the api endpoint so it automatically detects the incoming json structure.
Thanks for the instruction! I am able to get the endpoint recognizing the data converted from the plugin. I use the plugin to send a sample csv to a mock endpoint URL and then use Postman to send the JSON body back to the Bubble endpoint URL.
However, when I try to add that backend workflow, itās not showing up in the dropdownā¦
once you run the āConvert CSV to JSONā step then your data is sent to your ārunjsonā backend api workflow. You can then run any steps inside that runjson workflow to mutate your dataā¦
Thanks for your instruction! @shawnmi6
