Loading...

#DIBtraining

Pydoit Half Day Workshop

775 views

Loading...

Loading...

Rating is available when the video has been rented.
This feature is not available right now. Please try again later.
Streamed live on Jan 20, 2016

20:20 - lesson starts

Lesson materials here: http://www.camillescott.org/pydoit-in...
pydoit documentation: http://pydoit.org/

Twitter Hashtag: #DIBtraining

20:20 - Lesson starts, Introduction to doit
23:29 - local install trouble (skip this part)

24:59 - Topic 1 - BACKGROUND AND BASICS
25:10 - doit vs Make, what is doit
27:19 - example of simple pydoit task (Hello, world)

32:40 - Topic 2 - TARGETS
33:40 - reveal of origin of extremely very important data
34:24 - breakdown of download data task, actions and targets

39:18 - Topic 3 - DEPENDENCIES - how to not rerun every command every time
40:30 - File dependencies intro
40:58 - “uptodate” entry in task dictionary
46:20 - determining if a task is up to date or not
51:10 - putting file dependency in task in dodo.py
55:50 - Question: Are there other types of dependencies besides file dependencies?

57:20 - coffee break (skip this part)

1:11:30 - coffee break ends, wonderful sponsorship message from Camille

1:12:57 - Topic 3 - PYTHON TASKS
1:13:30 - python code inside tasks to create heat map from data
1:27:44 - Camille gets excited about this very important data set

1:29:40 - Topic 4 - CLEANING
1:34:40 - solution comes from Lex at 1:40:40 insert here
1:40:30 - break to work on adding attributes to existing tasks
1:40:40 - solving problem from earlier
1:43:00 - Camille adds clean attributes to existing tasks

1:45:25 - Topic 5 - SUBTASKS
1:48:10 - adding image file to document using jinja2 dynamically
1:49:20 - subtask breakdown
1:58:00 - showing jinja2 template

2:07:23 - Topic 6 - ADVANCED DOIT - creating application
2:10:45 - Giving arguments to tasks (first way)
2:11:30 - Using arg.parse (second way)
2:14:10 - Using python decorator to add new functionality to function
2:17:50 - Defining new task loader to overwrite general loader
2:29:30 - End of lesson material

2:30:35 - Adding titles to tasks
2:35:00 - Question - Does it matter which order tasks are in?
2:36:10 - Demo how to change order of tasks, enforcing ordering
2:40:36 - Question - For doit application, do you need to build the DAG yourself?
2:46:00 - Running processes in parallel, specifying threads
2:47:07 - Running specific commands at a time instead of all
2:50:00 - Getting info on specific tasks

Loading...


to add this to Watch Later

Add to

Loading playlists...