Recently I needed to load a CSV located in a public Amazon S3 bucket into Vertica. There’s some Vertica documentation for setting up a User Defined Load (UDL) Source, but here I’ll show you how to bring it all together.
In short, there’s a UDL Source for cURL in /opt/vertica/sdk/examples. You build the UDL from the given cpp code using make, and then create a library and source in Vertica for cURL. Finally, you can use cURL with a COPY statement to load data into a table.
4 years ago ·
Hi Sohan,
Thank you for the nice post. I have been trying to build cURL module, but unable to do so. Whenever I run make I get the following error :
c++ -I /opt/vertica/sdk/include -I HelperLibraries -g -Wall -Wno-unused-value -shared -fPIC -I -o /opt/vertica/sdk/examples/build/ScalarFunctions.so ScalarFunctions/*.cpp /opt/vertica/sdk/include/Vertica.cpp
clang: error: no such file or directory: ‘/opt/vertica/sdk/examples/build/ScalarFunctions.so’
make: *** [/opt/vertica/sdk/examples/build/ScalarFunctions.so] Error 1
I would really appreciate some help. Many thanks.
4 years ago ·
Hey Tariq,
Double check that you were able to compile some of the required cpp packages:
# see what packages you have built. You want ScalarFunctions.so to exist here
ls -l /opt/vertica/sdk/examples/build/
# compile some required packages
cd /opt/vertica/sdk/examples
make clean
make
Best,
Sohan