How to curl file to hadoop through Hoop using PHP -

How to curl file to hadoop through Hoop using PHP -

i want curl big file hoop using php. if normal php file upload there's headers prepended file.

when seek this:

$url = http://hoop:14000/filename?op=create&user.name=root $ch = curl_init(); curl_setopt($ch, curlopt_url, $url); curl_setopt($ch, curlopt_binarytransfer, true); curl_setopt($ch, curlopt_returntransfer, 1); curl_setopt($ch, curlopt_post, 1); curl_setopt($ch, curlopt_binarytransfer, true); curl_setopt($ch, curlopt_postfields, array("file" => "@" . $this->filepath)); curl_setopt($ch, curlopt_httpheader, array('content-type: application/octet-stream', 'expect:')); $content = curl_exec($ch);

the file on hoop have these headers:

------------------------------f0f063939ed8 content-disposition: form-data; name="file"; filename="phpbsa4ty" content-type: application/octet-stream {binary info here........}

i'm guessing needs raw post data. can work this:

$url = http://hoop:14000/filename?op=create&user.name=root $filedata = file_get_contents($this->filepath); $ch = curl_init(); curl_setopt($ch, curlopt_url, $url); curl_setopt($ch, curlopt_binarytransfer, true); curl_setopt($ch, curlopt_returntransfer, 1); curl_setopt($ch, curlopt_post, 1); curl_setopt($ch, curlopt_binarytransfer, true); curl_setopt($ch, curlopt_postfields, $filedata); curl_setopt($ch, curlopt_httpheader, array('content-type: application/octet-stream', 'expect:')); $content = curl_exec($ch);

but big files cause memory errors:

php fatal error: allowed memory size of 33554432 bytes exhausted (tried allocate 8388608 bytes)

is there way post raw files without loading file memory?

i can on command line using instructions hoop documentation

curl -x post -c ~/.hoopauth "http://<hoop_host>:14000/<path>?op=create[&<option>]*" \ --data-binary @data.txt --header "content-type: application/octet-stream"

from http://cloudera.github.com/hoop/docs/latest/httprestapi.html

php curl hadoop hdfs