upload data to galaxy using terminal

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

upload data to galaxy using terminal

do kadya
Hi,
Can I upload data (mostly link the data) using terminal ?

MuSiC 2 tool integration is not available in their website nor in tool shed.
MuSiC 2 creates folder and files by itself, which is hard to track by Galaxy.

So is there any way to link files to galaxy current work space so that I can link those distracted files which are created by MuSiC 2.

ex: if you run  
                music2   bmr     calc-bmr    file_of_input

output is 3 files,
./file1.txt
./folder1/file2.txt
./folder2/file3.txt
we can not mention that we need this output filename or save file at specific location, we can not give this option.


Galaxy will not able to track this output, hence i have to do it manually. So is it possible to upload this files manually by terminal ?




Waiting for positive reply.
Thank You
- Chetan

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload data to galaxy using terminal

Martin Čech
Hi,

yes, you can link files to a Galaxy Library without copying -  please see https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles (the last part 'Upload files from filesystem paths'). 

And there is an API for libraries (although it is under construction). The flow of controllers for upload is api/library_contents > controllers/library_common


Martin


On Tue, Mar 25, 2014 at 7:47 AM, do kadya <[hidden email]> wrote:
Hi,
Can I upload data (mostly link the data) using terminal ?

MuSiC 2 tool integration is not available in their website nor in tool shed.
MuSiC 2 creates folder and files by itself, which is hard to track by Galaxy.

So is there any way to link files to galaxy current work space so that I can link those distracted files which are created by MuSiC 2.

ex: if you run  
                music2   bmr     calc-bmr    file_of_input

output is 3 files,
./file1.txt
./folder1/file2.txt
./folder2/file3.txt
we can not mention that we need this output filename or save file at specific location, we can not give this option.


Galaxy will not able to track this output, hence i have to do it manually. So is it possible to upload this files manually by terminal ?




Waiting for positive reply.
Thank You
- Chetan

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/


___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload data to galaxy using terminal

do kadya
Thank you very much Martin Čech for reply.
Currently I am uploading data by using linking method only,
But what I wanted is NO API, that means via terminal only.

As you can see my first mail where I am explaining query, 
MuSiC2 creating folders and files by itself, and hence galaxy will not be able to track those files.
hence I want to collect those files and upload to current history (as data sets) using Terminal.
Is that thing is possible ?

else how can I tell Galaxy that tool is creating folder first and then writing files ?
I am using    http://cli-mate.lumc.nl/define#    to create xml file for galaxy.
I know I can hide output (from main command), but how can i track file which is inside folder ?

Waiting for positive reply.

On Tue, Mar 25, 2014 at 6:04 PM, Martin Čech <[hidden email]> wrote:
Hi,

yes, you can link files to a Galaxy Library without copying -  please see https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles (the last part 'Upload files from filesystem paths'). 

And there is an API for libraries (although it is under construction). The flow of controllers for upload is api/library_contents > controllers/library_common


Martin


On Tue, Mar 25, 2014 at 7:47 AM, do kadya <[hidden email]> wrote:
Hi,
Can I upload data (mostly link the data) using terminal ?

MuSiC 2 tool integration is not available in their website nor in tool shed.
MuSiC 2 creates folder and files by itself, which is hard to track by Galaxy.

So is there any way to link files to galaxy current work space so that I can link those distracted files which are created by MuSiC 2.

ex: if you run  
                music2   bmr     calc-bmr    file_of_input

output is 3 files,
./file1.txt
./folder1/file2.txt
./folder2/file3.txt
we can not mention that we need this output filename or save file at specific location, we can not give this option.


Galaxy will not able to track this output, hence i have to do it manually. So is it possible to upload this files manually by terminal ?




Waiting for positive reply.
Thank You
- Chetan

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/



___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/
Reply | Threaded
Open this post in threaded view
|

Re: upload data to galaxy using terminal

Martin Čech
I don't understand your question. What is stopping your from using the API from terminal? 

However the easiest way to do it would probably be with Python and the BioBlend project (http://bioblend.readthedocs.org/en/latest/ and https://github.com/afgane/bioblend).

Another way is to set up a watcher in the folder that your tool is creating files in and upload to Galaxy on change.

If none of these make sense to you please clarify your need.

thanks

Martin


On Wed, Mar 26, 2014 at 2:27 AM, do kadya <[hidden email]> wrote:
Thank you very much Martin Čech for reply.
Currently I am uploading data by using linking method only,
But what I wanted is NO API, that means via terminal only.

As you can see my first mail where I am explaining query, 
MuSiC2 creating folders and files by itself, and hence galaxy will not be able to track those files.
hence I want to collect those files and upload to current history (as data sets) using Terminal.
Is that thing is possible ?

else how can I tell Galaxy that tool is creating folder first and then writing files ?
I am using    http://cli-mate.lumc.nl/define#    to create xml file for galaxy.
I know I can hide output (from main command), but how can i track file which is inside folder ?

Waiting for positive reply.


On Tue, Mar 25, 2014 at 6:04 PM, Martin Čech <[hidden email]> wrote:
Hi,

yes, you can link files to a Galaxy Library without copying -  please see https://wiki.galaxyproject.org/Admin/DataLibraries/UploadingLibraryFiles (the last part 'Upload files from filesystem paths'). 

And there is an API for libraries (although it is under construction). The flow of controllers for upload is api/library_contents > controllers/library_common


Martin


On Tue, Mar 25, 2014 at 7:47 AM, do kadya <[hidden email]> wrote:
Hi,
Can I upload data (mostly link the data) using terminal ?

MuSiC 2 tool integration is not available in their website nor in tool shed.
MuSiC 2 creates folder and files by itself, which is hard to track by Galaxy.

So is there any way to link files to galaxy current work space so that I can link those distracted files which are created by MuSiC 2.

ex: if you run  
                music2   bmr     calc-bmr    file_of_input

output is 3 files,
./file1.txt
./folder1/file2.txt
./folder2/file3.txt
we can not mention that we need this output filename or save file at specific location, we can not give this option.


Galaxy will not able to track this output, hence i have to do it manually. So is it possible to upload this files manually by terminal ?




Waiting for positive reply.
Thank You
- Chetan

___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/




___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

  http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/