Part 1: GNU Parallel script processing and execution

[Romxero]

Im using the program and I love it! Is there a way I can use my distributed filesystem to access data for the tasks instead of passing it the way it does now? I use glusterFS and I just want to process in my mount

[drjnet1]

This is lovely stuff, thanks for sharing... think ill try this on my data mining grep script.

Is your time script available for borrowing?

[joaorstm]

@kalinin76 test -z checks if a string's length is zero.

In this case, the string being tested is probably user-configurable and can be empty.

This does not look like a mistake, and they also didn't mean test -d because mkdir does nothing if you try to create a directory that already exists.

[technicalsupport]

Thank you for creating what looks like an excellent tool, and for providing a fantastic tutorial. I'm looking forward to trying it out.

[rhl6856]

@kalinin76 it says check if that directory exists or make that directory if no error exists (the -p).

same as test -z /usr/local/bin && mkdir /usr/local/bin

[yingted]

@macemoneta

I think the focus is on the part where he used multiple machines. Anyways, xargs -P would mean that porting to parallel is even easier.

[macias102]

Yesterday, I started to convert my scripts to use parallel -- it is a great tool and so far conversion is smooth. Thank you very much!

[ntheory914]

@kalinin76 If /usr/local/bin doesn't exist then it will create that path (and all paths leading up to it) so it has a place to install GNU Parallel when it is done

[lonegroover]

Outstanding. Will be plumbing this into my MP3 conversion script today.

[crownstupid]

Looks like a possibly very useful tool. Can't wait to run across a real problem to use it on.

[TurboSmurfHDRemix]

Wonderful video. Well done.

[Eonflare]

@kalinin76 It tests to determine whether /usr/local/bin is there, and if it isn't it creates /usr/local/bin.

[BadVoidShip]

@kalinin76 It will check for the directories existence, if it doesn't exist it will create it.

The || is the OR operator, if the first condition is met, since it only needs 1 condition to be met, it won't bother to try the second condition (which is actually a command, but it returns an error code so it can be 'tested').

You can also use the AND operator && in a similar way, in that case the second command will only execute if the first one is successful.

[craftyguy1]

@kalinin76

Test if the /usr/local/bin directory exists, if it doesn't, use mkdir to create it (-p tells mkdir to create any parent directories in that path if they do not exist either)

[Sirupsen]

What is it with Unix and this cream colored background?

It's everywhere there's Unix!

[thomasknauth]

What shell are you using? Curious with respect to the file renaming syntax using {} and {.}.bz2

[OleTange]

@thomasknauth the {}, {.}, {3}, and {3.} are all GNU Parallel specific. It is not dependent on the shell.

[Racecar564]

GNU Parallel rocks!

[kalinin76]

@OleTange I'm just surprised, because

test -z /usr/local/bin

will always return false. test -z tests whether a string is empty. They probably meant test -d instead.

[macemoneta]

'xargs -P'?