how to merge multiple fastq files

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

how to merge multiple fastq files

Jen Hillman-Jackson
Hi Tracy,

The tool "Text Manipulation -> Concatenate datasets tail-to-head" can
put the two together. Then set the datatype to fastq, if needed (click
on the "pencil" icon for the full dataset to reach the "Edit Attributes"
form to make the change).

Be sure to watch out for an extra newline being added between the two
files. You can test for/eliminate this using "Filter and Sort -> Select"
with the options "that: NOT Matching" and "the pattern: ^$".
(the regular expression '^$' [no quotes] matches empty lines).

I am sending to the galaxy-user mailing list for our internal tracking
and for other users to learn from. Please send all questions directly to
the list going forward, it is very helpful for us.

Thanks for using Galaxy!

Jen
Galaxy team



On 10/27/11 11:58 AM, Qingquan Liu wrote:
 > Hi Jennifer,
 >
 > I have sequenced one sample on 2 lanes of GAII, and now I am trying to
 > merge the 2 fastq files. How should I do it on Galaxy? Thank you very
much!
 >
 > Best,
 >
 > Tracy
 >

--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support

Jennifer Hillman-Jackson
http://galaxyproject.org
Reply | Threaded
Open this post in threaded view
|

Re: how to merge multiple fastq files

Kevin-2
Hi Jennifer,
just out of curiousity, is the command a linux cat? or does it actually parse the fastq sequences?
asking as my service provider actually just split my PE files by lines (not no. of reads ) in fastq

assuming that I can't merge them in galaxy, I am merging them properly then splitting them again (anyway i can map using concurrent processes and merged the bam later on

Cheers
Kevin

On Fri, Oct 28, 2011 at 10:30 AM, Jennifer Jackson <[hidden email]> wrote:
Hi Tracy,

The tool "Text Manipulation -> Concatenate datasets tail-to-head" can put the two together. Then set the datatype to fastq, if needed (click on the "pencil" icon for the full dataset to reach the "Edit Attributes" form to make the change).

Be sure to watch out for an extra newline being added between the two files. You can test for/eliminate this using "Filter and Sort -> Select" with the options "that: NOT Matching" and "the pattern: ^$".
(the regular expression '^$' [no quotes] matches empty lines).

I am sending to the galaxy-user mailing list for our internal tracking and for other users to learn from. Please send all questions directly to the list going forward, it is very helpful for us.

Thanks for using Galaxy!

Jen
Galaxy team



On 10/27/11 11:58 AM, Qingquan Liu wrote:
> Hi Jennifer,
>
> I have sequenced one sample on 2 lanes of GAII, and now I am trying to
> merge the 2 fastq files. How should I do it on Galaxy? Thank you very much!
>
> Best,
>
> Tracy
>

--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support
___________________________________________________________
The Galaxy User list should be used for the discussion of
Galaxy analysis and other features on the public server
at usegalaxy.org.  Please keep all replies on the list by
using "reply all" in your mail client.  For discussion of
local Galaxy instances and the Galaxy source code, please
use the Galaxy Development list:

 http://lists.bx.psu.edu/listinfo/galaxy-dev

To manage your subscriptions to this and other Galaxy lists,
please use the interface at:

 http://lists.bx.psu.edu/

Reply | Threaded
Open this post in threaded view
|

Re: how to merge multiple fastq files

Jen Hillman-Jackson
Hello Kevin,

This particular function is like a "cat" - will work any text file.
Assuming that you run the function in the same order the data was split,
it should merge fine.

Should you want to work with tools that manipulate FASTQ files
specifically, a tool search of "FASTQ" will bring up most plus look
under the tool group "NGS: QC and manipulation -> Generic FASTQ
manipulation".

And in case you didn't see this, data can be uploaded using FTP. Several
compressed formats are support. Using a desktop client can the track
progress of, restart, and confirm a load.
http://galaxyproject.org/wiki/Learn/Upload%20via%20FTP

Best wishes for your project,

Jen
Galaxy team

On 10/27/11 9:01 PM, Kevin Lam wrote:

> Hi Jennifer,
> just out of curiousity, is the command a linux cat? or does it actually
> parse the fastq sequences?
> asking as my service provider actually just split my PE files by lines
> (not no. of reads ) in fastq
>
> assuming that I can't merge them in galaxy, I am merging them properly
> then splitting them again (anyway i can map using concurrent processes
> and merged the bam later on
>
> Cheers
> Kevin
>
> On Fri, Oct 28, 2011 at 10:30 AM, Jennifer Jackson <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Tracy,
>
>     The tool "Text Manipulation -> Concatenate datasets tail-to-head"
>     can put the two together. Then set the datatype to fastq, if needed
>     (click on the "pencil" icon for the full dataset to reach the "Edit
>     Attributes" form to make the change).
>
>     Be sure to watch out for an extra newline being added between the
>     two files. You can test for/eliminate this using "Filter and Sort ->
>     Select" with the options "that: NOT Matching" and "the pattern: ^$".
>     (the regular expression '^$' [no quotes] matches empty lines).
>
>     I am sending to the galaxy-user mailing list for our internal
>     tracking and for other users to learn from. Please send all
>     questions directly to the list going forward, it is very helpful for us.
>
>     Thanks for using Galaxy!
>
>     Jen
>     Galaxy team
>
>
>
>     On 10/27/11 11:58 AM, Qingquan Liu wrote:
>      > Hi Jennifer,
>      >
>      > I have sequenced one sample on 2 lanes of GAII, and now I am
>     trying to
>      > merge the 2 fastq files. How should I do it on Galaxy? Thank you
>     very much!
>      >
>      > Best,
>      >
>      > Tracy
>      >
>
>     --
>     Jennifer Jackson
>     http://usegalaxy.org
>     http://galaxyproject.org/wiki/__Support
>     <http://galaxyproject.org/wiki/Support>
>     _____________________________________________________________
>     The Galaxy User list should be used for the discussion of
>     Galaxy analysis and other features on the public server
>     at usegalaxy.org <http://usegalaxy.org>.  Please keep all replies on
>     the list by
>     using "reply all" in your mail client.  For discussion of
>     local Galaxy instances and the Galaxy source code, please
>     use the Galaxy Development list:
>
>     http://lists.bx.psu.edu/__listinfo/galaxy-dev
>     <http://lists.bx.psu.edu/listinfo/galaxy-dev>
>
>     To manage your subscriptions to this and other Galaxy lists,
>     please use the interface at:
>
>     http://lists.bx.psu.edu/
>
>

--
Jennifer Jackson
http://usegalaxy.org
http://galaxyproject.org/wiki/Support

Jennifer Hillman-Jackson
http://galaxyproject.org