 नमस्ते, अपका इस नेक्स्लो और अन्प्कोर समवदाय दवारा आयोजेत अन्लाईं प्र्सिक्ष्यम ने फिर से स्वागत है, आज हम दुसरे स्टेशिन के साथ इस ट्रेनिंग को कुन्तिनुग करेंग है, और, कुछ मेरे बारे में, मेरे नाम अबिनक्फ शर्मा है, और फिलहाल मेरी तो अफिलेवशन्स हैं, पहली पहली फींज्द स्टेलिन बाश एनूववेर सी चाँत अफरिका में. और अज यस के साथ साथ मैं भायन्प्रमाटिक्स वन्शल टिन्ची अप्र करता हूं, को इक सरल और कम्फुट लंगुज में ट्रास्लेट करके लोगं तक पुचाएं इसके साथ हम कल के एक तोड़ सरीक अप करते हैं सेशिन वान कल की मेजर ठीम ती हमारी नेक्ष्लो बेसिक्स अप आप आप आप तक ये खिल तुश्वाँ शाथ कर था हैं इसके बाद हमने नेक्ष्लो की इक चोटी सी श्क्रिप्त के साथ काम करागा फ़ा हलो दोड नेक्ष्ष्क तुशके फ़ुट कुछ कोंषेप्त को अई टुच करागा था औसके बाद हम ने श्टेप पी श्टेप तरीके से एक आ़े नेशीख पाईप्लिन को धुपलप करागा था दो अगर आप उन्ट कुन्ष्ट से क्मफतिबल है तो डेफनेतली हम आगे बडते हैं मैं आई उजित की है, इसले हमारे पाज शेप्रेट चानल है। हर इक चानल में कुछ वोलन्टीर्स है। जो आपके सवालो का जबाब देने की पुरी कोछिष करेंगे इसके साथ, हम आज के सेश्यम की शुवात करने के रहे है। आज के लिए हम एन्द्फकोर की वेप्साइट से श़्ाथ करेंगे जिसका लिएक है, एनद्फकोर ुट, एनद्फको ुट अर्ए तो आए ए, चताथ करते है। जैसे ही, आप एनद्फकोर की वेप्साइट को अपने ब्राद़र पे कोलनेगे विल्ड़ा क़़र का हो यो लग्ट बैग़़़़़्््। वो आपकी अम्पुटर की फीचे थीम पे धिपन करता है मेरे लिए बैग़़़़््््ार्ख है लिकिम ख़दी बिल्खोल सेईम है तो आप देक सकते है कि एप आन्वकोर की जो ताग लैंया है वो है, a community effort to collect a curated set of analysis pipelines built using Nextflow. तो यहांपे मैं, highlight करना चाहोंगा की, community NF core मैं, major role play करती है, और, community का goal यह है की वो, analysis, high quality, analysis pipelines को, develop कर सके, curate कर सके, Nextflow का, use करते होगे. तो आई यह देकते हैं की, हम, NF core की, functionality को और जो तूलिंग है, वो से कैसे कैसे use कर सकते है, NF core जो है, facilities के लिए, कैसे help कर सकता है, जैसे अगर आपकी को, university है, आपकी से company में काम करते है, तो, उन organizations के लिए, NF core कैसे help कर सकता है, की जो NF core की सारी pipelines है, तो अगर चार से PU चाई ये उस प्द्लाम, या उस प्रोस्ट्स को रन करने कि लिए, तो ओप्तमाइस ते है, इसके साथ साथ वो पाइपलान, exlent reporting of करती है, हरेक NF core पाइपलान exlent reporting of करती है, ताके एन में आपके पास इक सिंगल रिबोट आए जिस में आप जादतर अपने पुरे पाइपलन के एकजिकूशन और सामपूल्स के अनालिसस को समराइस कर सकें इसके साथ हम हरेक रिएक को वलड़ेट करते हैं मैं इसके बारे में और बात करूगा अगे आप यह पाइपलन की खुशके यह पाइपलन की कोई स्टेबल रिग लिज होती है जैसे की आहा लिख है, स्टेबल रिग रिग होती है गिटव पर, गिटव हमारा आन्प्कोर का वर्जिम खुब चूल बाँनच्मट अप रोस्तिण सलुशन है तो जो भी code-related activity होती होती है, तो गिटहब पे है जैसे ही गिटहप पे एक पाइप्लान की stable release आती है, automatic tests trigger होते है, ताके हम ये देख सके हैं, तो इसके साथ साथ पाकेज सोफ्रेर, अनेप्कोर की पूरी कोषीच है, कि वो जो टेकनलोगिकल अदवान्सेज हैं, सोफ्रेर अनेप्कोर लेके उंगा पूरी तरे से अस्तमाल करे, इसका मतलब यह एक कोंडा और कंटेनर सिस्तम्स like इसके साथ साथ पोरटेभोल अर्पीडोऩवड्यए groom ड़सीच कि �得गे साथ साथ में ढ़ोगी उरतीने आगमी रह में �しか muita of it, now. यह के पीजिज में है यह एक एह आनप्फोर कमुडिटी के अनदर है, वो पाईप्लाईं जो है बेस प्रक्टिजिस को फुलो करे, और वो हनेद पुर्सेः रीभ्रुटिस़़वल ओब और फोटेबल हो. किंकी अगर हम best practices नहीं फोलो करते, तो हो सकता है कि जो hardware या infrastructure का जो logic है, configuration का logic है, वो overlapping है, pipeline के logic से, तो ये अच्छी बात नहीं है किंकी, अब अगर ये दोने चीजे same level पे हैं, तो उस pipeline को अगर अप दुसरे infrastructure पे रन करना चाहें, या कोई तुसरे collaborators अपने infrastructure पे रन करना चाहें, तो वो नहीं कर सकते हैं, इसले best practices बहुत सरोडी है, इसके साथ साथ, pipelines की releases को हम cloud पे भी टेस करते हैं, AWS, Azure, Google Cloud, तो जैसे ही कोई stable release आती है, तो हम से बहुत सारे environments में तेस करते हैं, चोटे data sets के साथ, और बडे data sets के साथ, सब कोछ हमारी पुरी कोछेच है, कि हम से end to end कवर करें, देवलेपर्स और कम्मिनूटी, NfCore का मेजर फोकस है, NfCore खाली पाईप्लान की रजिस्टी नहीं है, जैसे की डोकर रजिस्टी होगें, यह फिर कोड रजिस्टी होगें, NfCore, 100% driven है, कम्मिनूटी प्रोसिसे, और हमारी पुरी कोछेच है, कि हम, कोलाबरेशियंस, इस कम्मिनूटी की आंदर फसिलिटेट करें, जिस का मतलब यह है, कि अगर लोग, इस कम्मिनूटी में लोग कोलाबरेट करना चाते है, ता कि गरुप A अगरुप B दोनो एकी पाईपलन को यूस करें, चाहे वो दिख्रन प्रामिटेट के सात हो, तो ये एक मेजे फुकस है, कोलाबरेशियं का हमारा, इस कि सात मैं यह हैलेट करना चाहोंगा, कि NfCore, website पे और YouTube चानल्प पे खासकर, बहुत सारे वीटियोंगे है, जो और लग लग कोंटेक्स को कवर करते हैं, जैसे एक पाईट साइस तोक होगया, जो शोड तोक्स होती हैं, 15 मिनेट्स, 20 मिनेट्स, कुष तोट्योरिल ताईप के वीटियोंगे हैं, यह आप चात साथ पनेद्स भीटियोंगे है, तो अगर आप इन वीटियोंगे दिखने में अप टीज़ेट आप आप जैए इन दीवटियोंगे हैं, तो अप यह तेपिनेट्गे एप कोर के यूटिप चानल्ट पर चाईगे, वहाँपे आपको इन सब वीटोँ का अख्सेस वहाँ स्वोग जाएग़। इसके साथ अगर आपके पास आपके कमपुटर में क्र्ल या नेक्ष्फ्लो औजावा पाइट्तिन पेप पहले से अंस्टल्ड है तो आप आपके लिए ग्टिंग साथर बहुत एजी है आपको बस नेक्ष्फ्लों अंस्टल करना है और आप कोई भी आन्प्कोर की पाइप् अंस्टल के साथ इस सिंगल कमबडलान तूल के जर्या आप अप अप अप अप अप अप अप आपको रहा है जाँगा की बहुत साथर इसे हैं दुनीया में जो आपकोर की पाइप्लाइण का यूज करते हैं जैसे नुयोग एन औब स्टीटी इस्टॉट पास्टर अप आ अप भत रता है इसात क arsenal दुसी सैक्चें स्रह कilty अप आप प्कलाइब य्यर हूँँँओ है उऄह कुँँँई उशन अप आबमाए जी लगृँँ पास्ट्ट्ट delle आप कegh based blue renew याप फिर उंका जो कोंटेक्स था वो दुसरी पीप्लाइन लिए बाख़ी ने कवर रव कर लिया है. यासे ही इस पीप्लाइन का स्पेस्विक पेज कुलता है, हम देख सकते है है कि उस में और भी ताप सवेलबल हैं. जैसे इंट्रॉट्ट्चन, रीजाल्ट, युसेज एक सेट्रा है, हम ताप सको अभी कवर करेंगे. असे बहफले में या रहलेट करन रहाता है हूं, कि जैसे ही पीप्लाइन का पेज कुलेगा उस पीप्लाइन की ताग्लाइन आपको दिखेगी, और सके साथ साथ आप यी लज़् परटिन देख सकते हैं, ये अन्लीग कोर तूल्ट के साथ लाआँँथ है, लेक्न इक पाप्लणों को लोंज करने के और भी तरीके है। टेखने मैं आभी कवर कर उंगा इसके साथ-साथ मैं यह लेड करने चाता हों के अगर अब बआग अगर अप पिछले वोर्जन को कवर करने चाऑई आया आप पिछले अप देख सकते हैं कि इस पाप्लान के बार में एक ब्रीख अंट्रॉट्चन है उसके नीचे हमने इसके तोड़ी से आस्टी भी बताए है उसके साथ हमने पाप्लान समरी कवर करी है ताकि उस समरी के अंडर एक परटिकुलर स्टेप में तोड़ी समरी के अंडर सब जल्दी से विसिबल हो इसके साथ साथ अप रहीट हैंट साथ पर यह देख सकते हैं के और भी अप्षन सवेलबल हैं कि अगर अप इस पाप्लान को अन्ट्टॉल्ट्चन के साथ रन करना चाहें तो कैसे कर सकते हैं तो अब यह दीख्रमप मुझे जो भी टेख्स्टे डीख्रम में मुझे अच्छे से दिख्रा हैं यहां पे अप फुंक्षनालती अवर्विव देख सकते हैं हमरी पूरी कोशिष हैं के हर एक पाप्लान के पास यह यस तरेक तरनल डीख्रम अवलेबल हो अच्छे साथ रफ औज़़़ से पारेप्लान को येसट प्रोफय लध कर सकते हैं अगर आप अप अ отлич़ कगत का डीट cosplay उजित ःगत का छाते हैं तो अप को मुछी सज मुआर पास में यह बात को से रफ डाओ न मुखेखृ अप अप अप आप पिप्लान की सम्पल चीट के बारई में अन्हां ले सकते हैं और कुई स्टप्स को अगर अप श्किप करना चाते हैं तो वो इन्फुमशिन भी यह अप यह आप यह आप परे लैएडल हैं इस के साथ आप यह देख सकते हैं अप इसे आजवर पे कैसे चलाना चाहा हैं यह फिर एक पाईप्रन्त को बैग्रोंग में कैसे चला सकते हैं उसके बारे में भी हमने अन्पुमेशन आद करी हैं इसके साथ हम अगले ताप की तरव बड़ते हैं तोगी है, परामेटर्स आभ दीख सकते हैं कि साथ हैं तोगी वि सब कुछ इस पेश्पेष्ट बाईबल हैं जैसे की इस पाआईप्लन् का नजो सीख पाईप्लन् के पास से एक इनबुट परामेटर है जो जाथतर अनफ कोर पाईप्लाइन्स में है इसकी दीपाईट्फलु यह है नानोसीग प्राईप्लान के पास एक इनबुट प्रामीटर है, जो जादतर अनिप्कोर प्राईप्लान्स में है, इसकी दीफाल्ट लेए है, लेकिन अगर अगर अप इसके बारे में कुछ इन्प्रमेशन लेना जाते हैं, अपको इसकी इन्प्रमेशन में जाएगी, मैं ये भी, ये भी हालेट करना चाहूँँँँँँँ, की रेक्ष्टाग जिन प्रारमेटर्स पे लिखाया है, अगर वो प्रवाइट नहीं करे गए, और अप तेस्ट प्रुफाईल नहीं लेए है, तो ईस ईस आबच्टिप, यस पारामटर्टिठ गर तो आप जो आप अपने, इस पारप्टलेन को आपने ड़ेटा के लिए, और इन्प्राएटर्टटच्टर के लिए कस्टमाएस कर सकते हैं. कुईज परामेटर् सकए अमन है, तो नोरमली चेंच करने की जरूत नहीं है, लेकिन अगर अप चाहें, तो उसे अप चेंज भी कर सकते हैं। इसके साथ हम अगले ताप की तरव परते हैं। जो के है अगपुट तो हरेक पाईपलाईन का अउपुट क्योंके हर पाईपलाईन एक सबसेपक परपूट से नहीं करीगे लिए है, उसका अउपुट बहुट बहुट होगा तो आप ये देख सकते हैं कि यहां पेज अउपुट ताप हैं, उसके अंदर हम नहीं पूरी कोषिष करीग है, कि हम ये अप पाईपलाईन का अउपुट क्या होगा, किस जगेगे पे होगा, और उसकी इंटिप्रटेटेशन आप कैसे गर सकते हैं. तो अगर अप ये समझन चाते हैं कि किसी भी आनपकोर पाईपलाईन के अउपुट के अंटिप्रेट कैसे करना है, अप के लिए ताब बहुत स्रूटी है जैसे अपुट तोक्स, इसके साथ एक लास्ट ताब है हमारे पास जो की है, लिएज और स्टर्टिस्टिक्स के बारे में, आप ये देख सकते हैं, कि पिछली रिलीज कब हुई ती एक पाईपलाईन की, याफ यर आप उसे गडिट़़ पे देख सकते हैं, अगर अगर आप चाइच तो उसके सटेटिस्टिक सब देख सकते हैं, पाईपलिन के, और राट ठेएँच साथ पे आप उस पाईपलाईन का एक अगर कोpe video available हैं अप उसे भी शाथे न प्रिक सकते हैं, आप देख सकते हैं कि आपको लिस देखता है, तोटल मुड्योल्स कि तनेआवण्लग़ हैं, तो फिलहाल आजकी देप में 832 मुड्योल्स अवएलिटल हैं, अगर आप दो या तीन मोडूल्स को हमेशा यूस करते हो, तो उसका एक वोग्फलो बनाके एक उसे एक मोडूल के तरे टीट कर सकते हैं. इसका मतलब जैस से अगर आप बायन्फमाटिक्स दूमें का थोड़ एक सपीरिडिन्स रकते हैं, तो बात करते हैं, या फिर अगर अगर Nfcore सब वोग्फलोस प्रोजेक में वो मोडूल पहले से अप रेलबल हैं, तो आप उसे ड़ूल्ट कर के आपनी पाया प्लैन में यूस कर सकते हैं. हम इसके बाहरे में अप फिर से बात करेंगे प्रक्तिकल तूमस में, जब हम Nfcore तूल्ट को और अच्छे से देखेंगे, तो इसके साथ हम आगे बडते हैं, तूल्ट के एक चोटा साई अईंट्रूटक्षिन, तो Nfcore की तूलिंग है, वो बहुत ही इंदेप्त है, तूल्ट के अईंटेप्त है, और उसके डोकमेंटेशिन की भी डोकमेंटेशिन को भी इंदेप्त रखने की हमारी पूरी कुचिष है, जब से अब देख सकते हैं, हम ने पहले बात करे थी कि आप Nfcore को तूल्ट को और अच्छे से कर सकते हैं, एक एक में देखे आप सब वोग्फलों बना सकते हैं, जैसे सिमले तूल्ट मुड्फलों बना सकते है, जै मौडूल भी नonomy सल कर सकते है, ने आब मौडूल बना सकते है, तो ये सारे आप बिलटीख जो है आप के पहले लेबलवेल होती हैं, इसके इस में Nfco और माग पाइपलान के उपर ती ही तोग तो आप इसे देख सकते हैं इसके बारे में इन्फुमेशन अवलबल है अथर कों ता और इसका जो भी अबट्पुट है जो भीट्व है जंएली वो भी आप इस पीज पे देख सकते हैं अगर आने लग एवरेण्ट को देखना चाँई तोब देक सकते हैं के एवरे जाएवाण कहाईपिए हुइ और इसा अप देख सकते हैं अप जुँ पी ताएमाँ जोन में हों उस तायम्जोन के इसाजब से हमारी विब साइट बख्यट गगे करेगी और अपको उसके टाईम्जोन के इजाँप्से और इवन्थ के इजाँप्से आपको अपटेट्स देगीज। ज़योभीज इबेन्ट़ का उण्ट्ट्ट यह एह कोल है तो आप देक सकते हैं की यहाप एक एटिट बट्टन भी है और यह एन एप प्कोर की रग पेज्पट एटिट बट्टन अवेलग़ लेए आगर आप कोई टिस्टेक देकते हैं की कोई ताइपो हैं यह थो आप फर आप गगे अप नेए कुड़ेंत आद करना चाते हैं तो आप भन लेग अप आदें कर सकते हैं तो आप एक आप अदिन उड़ें दबाईएं और गेटुप पे उस आप फैल को चेंज करें अगे और एक पुल गेखेष करें अपका कुन्त्रबूँशन बहुत वालूँट है इसके साथ मैं आपको यह आपको यह लेड करना चात्ता हूँ कि हमारे पास जो है एक अबाउट पेज है जिस मैं आप एन्प्कोर कमुडिटी के बारे में देख सकते हैं आपको करन्ट्रबूँटी का एक पेज हैं जो किसके अब खोल मैं देख सकते हैगे जेल्गी से इस कमुडिटी इस पेज पर आप यह देख सकते हैं क्युटिएए काँं खान्ट्रिष काहां से हैं अग्के वोंने किनी जेल्गट देख सकते हैं प्रिल्हाँ आज़ की लेट में ने प्कोर के फ्लाक मैंबस, फ्वाजँँ साँईच मैंबर से हैं। और गेट अप ब हँछ चारे कन्त्रीबुटर से हैं। तो यह पेज तोर आँस ताई में लेए सकते हैं लोडवन में तो उसे साईट पीक कर सकते हैं जैसे कोप्रोएड, सर्कलर, आंप्ली सीख पाप्लाईन तो सारे पब्लिकेचन्स आपको एक सिंगल पेच्पे अवेलोबल हैं इसके साथ मैं कन्कलुट तना चाहूँवाग ये कहे कि आप इस कमुनेटी को जोईं कर सकते हैं एम लिए आपको कर के बाब रही है बाब रही है नहीं अप स्वादी पापिलानच को से यहा है ल्फ बाब रही है, अप नहीं ज़ो वहाग चाहूँँग, नहीं पैपलाइनाच कर कोई तना प्ली वाग अपनी आप अपंदक पेएके और रुकात करते हैं। ये देख सकेंगे शाइद आप पहले पोट्ट को बी ब�チャद करते हैं आप हमार पास जादा एरया है तो हम युँँँँँँँँँ पुट्ट करते हैं। तो हम इसके साथ कंटिनूँ कर सकते हैं उस में कोई प्रोडम नहीं हैं मैं बस यह एंचोर करनाच चाहूंगा की हमारा जो वोरिजन है, वो हम कनफर्म कर सकें जो की है, 2.7.2 और अगर नेक्स फलो का वोरिजन हम कनफर्म कर सकें तो हमारा इंवायमेड वो कोई जेडी है आप उन्गर नाम करत लिखा, लिखर बाज़ी रहीं लिए ती डाल अज्गा ना था तो इसके साथ हम देक सकते हैं, नेक्स भोका 22.10.1 वोरिजन है यह आप येगे और अव्कोर का 2.7.2 वोरिजन अवोलेबल हैं अगर अप वापस इस्प्लोरग कोगलें लेवट हैं सथ तो आप देख सकते हैं की यहापे पहले से बगत सब फुलरे जाएलगागें लेकिन ये एंनेप कोगलें तूल्स की लिए देवल्प्मेंट रिपोजट्ची अगद तो हम उस में चेंज नहीं खाँते हैं इस प्रोसेस में जाजा ताहिम नहीं लगना जाएगी क्योंकी गिटपोड आपका पहले से यह आपका एपका एज़े तेमठरी कमपुटर है पहले से यह रन कर रहें तो आपको बस उस फुलर्डर को कोगलें जाएगे जाएगे उस बाहरि कोगलें बज़कोर की तूल्स के सात हम कोगट को फमilirize करेई से रेटी हैं हमारा पहले स्टेप हो गए एन्फ कोर की ह spreading the help of the NF core की help कोगट करनेगा जेसे अगर अगकर आप, Nf core dash H तबाते है, तो आपको दिखेगेगा So you will see the help menu of NFCore tools, I will show the section here. May be this is it, it will be easier. If you look at the usage of NFCore, you can see options, command, and arguments. So you will have to see which argument or option you want to use first, or which command you want to use. कि यट्रिट मिरझसे क्यट अगर ती लिसत में खलिर है कि यट्रिट यट्रिट में आप आप ऐक है अपकों जे रिए लिस लिस कर ग़ावगा धो भी लिए फैए में खलिर कम देकpe जी लिस बना य स्प्रिलिंजाon यह नहीं कि लास्ट पूलट यह आपके एंवायमेंटप देपंट पर देपनटा है आभी मैंने कोई भी पाईप्लन डिस्टल नहीं करी यह इस एंवायमेंटप आप पूल लेए एगे लेएगे आप लेटेस द्रिलीज भी खाली है लिकं आगी अए एक प्लिप्लाण ड disag दिखतां क dels const आपनी ऍुल्चा देखिन मिल रहा यहाँ आपे अज्ट्कि बाद मैट्त्रेनींके पूल्लर के आंडल अंधर सिंगलारटिक अग्ट्काई पुल्डर बनावगा जवापे मैए से तो ज़िलिए हम अबही कि लिए, नण्कमप्रेशिन के साज ड़़़्ूट करते हैं और आम देख सकते हैं कि पहले पाईप्लटेन ड़़़़ूट होई, पाईप्लेन का वोईजन रभ पता चला तो इसे हमें विजोल फीटबाग भी मिलता है तो हम देख सकते हैं कि यह सब पारलेल में डानलोड हो रहा है जैसे ही एक कंटेनर डानलोड होता है NFO, tools, दूसरे कंटेनर को डानलोड करना स्थाद कर दिता है यह सब अतमाटिकली मनेच्ट है और अगर आप एखट्बलोरगक्हुलें तो आप शाठ देख सकें गे कुन-कुईन्से कंटेनर से तानलोड होतेंगे अभी किटपरट री रोड कर रहा है और यह सब कंटेनर से तानलोड होझेगे है हमारे, तमलरी कमँठ्च्ट आंतर आजके है ज़ियाख पार। एक उऴहां। और बल क्ये नहीं। ख़िए णभ काप की लगांगो। गर के फिल्टर दल कर कि के लिए और ग़ों ड्याख वह लेस्ट में। आप लिएलिस्ट लिएश्ट कर थाज्बिट रेएश्टाद अगर हम उपर जाके देखें इस तरमनल को गिट पोट पे स्क्रूल करें तो हम देख सकते हैं कि हमने कुन्सी कमाड यहाँ भे दीती जैसे Nfco Download तो Nfco Download में हमने DROP DOWNS को यूज कर के स्पैसिपाई कराता जैसे ही PIPELINE DOWNLOAD होगी जो के हमारे केस में SINGULARITY CONTAINERS के बाद ही हो सकता है हो गया है वो काम कमप्लिट और हमारे SINGULARITY CONTAINERS अगर हम TRAINING के अंदर जाएं मैंने चोटी सी गलती करी यहाँ पे कि जब मैंने नहीं तरमनल कोला था तब मैं वापस तूलस फुल्टर के अंदर आगया लेकिन कोई बात नहीं मैं यह यह मुप कर सकता हूँ आहाँ से अपने TRAINING फुल्टर के अंदर तो अगर हम TRAINING फुल्टर के अंदर जाएं तब हम देक सकते हैं कि जो भी SINGULARITY CONTAINERS ते अगर हम अंदर जाएं इस फुल्टर के तो हम यह देक सकते हैं की कुन कुन से चीजा अप़िलबिल हैं CONFIGS, SINGULARITY Images, और Workflow तो अगर में यहाँ पे TREE कमान देता हूँ हम देक सकते हैं कि क्या के अपिलबिल है यहाँपे भी हमार पा SINGULARITY Images के कोपिये अब्ऱिलबिल है और जो हमार वोग्फिलो का actual code है मोडूल़ सब चीजो के साथ वो हमें यहाँपे अबगिलबिल है लेकिन इसके साथ साथ आप एक पुल्टर देकेंगे CONFIGS नाम का जो की शहाथ आप अपनी रही लिए दिकरा है की बूपाईप्लाई जो है है वहाँभी लोकली पूल कर रही है तो, हम नहीं दिखा की आप एक पाईप्लाईं को लोकल दाओडूड के से कर सकते हैं और दूस्च्रा जो कमप्लीटली अप्लाईन विसेज के लिए अईडिल है तो दूस्रा आप एक एक अनलैन यूजज या अंटिनेट एनबेल्ट कमपुटर में नेक्स फलो का देरेक्ट यूजग के से कर सकते हैं अग पाईप्लान को डाउनलोट करनेक लिए आब हम अन्फ को तूलिंग की बात कर रहे हैं अब हम जल्दी से ये देकते हैं के एक डाउनलोड़द पाईप्लान को आप लांच कै से कर सकते हैं अगर आब दरेक्ट लिए नेक्स फलो का यूजग कर नाचाएं तो जैसे के हम नहीं आप आप आप आप आप आप आप प्लाईट को डाउनलोट कर रहा था अप उसे के सकते हैं, simple nxflow run nxcore rnaseek अगर आब dash r option या revision option अप़ी दीप्फलो को नेक्स फलो को नहीं बताते है तो nextflow remote या remote का मतलब है, Github पर जाकर जैसे मैंने nxcore rnaseek के लिखा, अगर मैं github.com अगर में ठुरा ये लिकुं। योरल तो ये आगर में सिर्फ अगर में, इपना पाड पतां। अगर में nextflow को सिर्फ इतना हिंट देता हूँ तो भी nextflow समजगगा की by default ये github की github की nfcore organization में saved pipeline है अगर अगर आप दुस्रा version control, hosting provider यूँस कर रहें जैसे pitpocket ये kitlab तो आप उसे भी इस तरे से use कर सकते हैं यहां पर आखके आप एक outdir को अपको जरूरी बताना होगगा क्यों की rnseek pipeline या कुछ nfcore pipelines यह आपको एक तरे से outdir को required parameter के तरे use करती हैं अभी में आपको अपको अपको अगर की आप कैसे discover कर सकते हैं कमान लैंपे कुँन से required parameters हैं कुँन से नहीं अब है development का का ता इसगे है। तरे से of choice के यह तरे आपको आपको आपको.. की .. तो शाएद हम कुछ प्रिकमपुटर तास्तो यूस कर सकें लेकिन नेक्स फलो हमें यहापे बोल्डर के नहीं वोर्जिन इसका मतलब हमें यहे से पूल करना होगा पूल करनेक लिए हम सिमिलर कमाड यूस करेंगे नेक्फलो की अगेन प्रिक्स शे पूल और इस बार हमने जासे ही पूल करा शाए दे वोर्जिन इक्चिस नहीं करता मैं गलत वोर्जिन लुस कर रहा होगा जल्दी से ख़फ्म करते हैं कुन कुअन से वोर्जिन है आप 3.10 तो 0 वोर्जिन कोई नहीं है इस लिए next-hole पूल नहीं कर सकता कुन सा वोर्जिन यह पिक करता है आप अप अप अप यह देखें की R&C का अभी करेंट वोर्जिन है जो हमने next-hole के पूल करा है Nfco tooling तूलिंग नहीं वोर्जिन पिक कर लिए अप को यह भी बता रहा है कि यह करेंट वोर्जिन यह लेटिस वोर्जिन नहीं है तो आप चाहें तो यह जूस कर सकते हैं directli अब हम अपनी वापस next-hole कमान के तरव जाएंगे और इक स्पैसपिक पिक पूल करेंट वोर्जिन के साथ दिस जुम करेंगे प्रीविस रन को हो सकता है कि कुछ प्रोसेस चेंज हुए हो उसकता है कुछ प्रोसेस चेंज हो हो तो next-hole यह इंटिलिजेंच पीहेवर यहाप को दिकायगा और जो भी प्रोसेस चेंज नहींब प्रोसेस रन नहीं करेगा यह वोर्टनिंग से यह जेंगे आपको प्रोबलम नहीं देंगी लेकिन अगर अगर यह प्रिंट होती हैं तो हमारे एकविनडेचन है क्या अप यह से अगर अगर अप आप अप अपने प्यप्लान के को अच्ठेसे समजतन चाते हैं तो आप ए एक बार उन वारनें मेसचिचस को अच्ठेसे पड़ें औग जोब भी लिएग समने अद कर हैं योंकी उस प्यप्लाइन का जब नया वुर्जन आया तो उसने पतिकुलर प्रोसेस को अप्टेट नहीं करा और नेक्स फलो इने ये लिए रेलाइस कर के हमारी कैस को अंटक्त रख है. अब मैं येसे कैंसल करता हों फिर से क्योंकी हम चाते हैं की कंत्रोल सीक के सात कैंसल और इस बार हम कुछ अलक ट्रीए करते हैं इस बार हम ट्रीए करते हैं लोकल पात को क्योंकी आपको शाए द्याद हो कि हमने अरेने सीक प्यप्लाइन को लोकली ड़ूड कर रख है पहले लेकिन वो नाँन सेंट्रल ड़ेटबेस एर्या में ड़ूड हुए तो इस लिए हम इसे पुरा पात देंगे काहाँपे हमने प्यप्लाइन को सेव करा है जिसे त्रेनिंग उसके बाद अनेफ कोर अरेने सीक वान इसके अंदर आप इसे वोक्फलो के सात लाके सेव कर सकते हैं और यहाँपे अप जो अनेफ को तूलिंग है यह यह क्या कर रही है? यह वह वह स्पिप्लटन के सखीमा को रेफर करता है सकीमा के सात मैं आप को उस्पिल्लन का तोड़ा कोड के सात वोगर भी दिख़ान जाता हूँँ आपकर हों R&E seek आखके इस पे क्लिख करते हैं गिटव्ड़ाँ एक आप को गिटव्ट़ाँ की लोकेशन पे लेजाएगा वो आप एक यह आप एक सकते हैं की लेटेस रिलेज गब की ती इस पैप्लैंकी, और इस पैप्लैंका इस ट्रक्छर जो है फाड़े क अंपिकुर प अगर अपको अच्ठह से समौचन मैं है, की जो नहीं जैं सुल्द अत्गिर सूए, अप उस कैसे लिक सकते हैं मैं अवह अपको लोग दिखाँगा. लक्न अव में अपको अप की पाह्बी लैंकी अपको अपको अच्चे सुमचन अहैं, कही कौए आप की नक्द कोड को जो करणा, आप साच्टा उब अब तरहए शाउं्छा बसबबर तरहते है, तो तो उगजर लिए साचआत नने लेखन हैं, क्या एक श्टके तो डिस तो तेखाए या एक रफा ठी तो डीक शाच्टा कोड कोड कंखाछिं, थे लेंगे बचा � Singularity, Shifter, these are all Container Images, Container Platforms. With this, there is a GITPORT profile, which we can use to run our pipeline. And there are two profiles, Test and Testful, which are related to the test data. If you just write a test and merge it with the Docker profile, this means that you are telling the next flow to merge the test and Docker profile and you want to run that pipeline with it. But if you look at NFCore's patterns, then Test and Testful profiles are not related to the hardware, they are related to the data. This means that the test is a mini or small-scale dataset. Testful means full-scale dataset, which we use to test this pipeline with every release of this pipeline. With this, I want to show you more other topics here. Maybe this option is coming in your Github on the left. If it is not coming, then you can go to the top level again. And you can see other folders from there. We will talk about which folder. You can see what configurations are available in which folder. You can see some configurations here, which we saw in the nextflow.config. Like Base.config, iGenomes.config, Modules.config, this is very relevant for us. Along with this, Test.config and Testful.config. You can open this file and see the content of this file. As soon as you enable the test profile, it overwrites the default parameters with its own parameters and gives a specific argument for a process. But if you look at Testful, there will be similar content here as well. Because we just want to change the input and output. And most of the time we don't want to touch any configuration with Testful and Testful profiles. Along with this, we go back to our top level. And I want to show you the nextflow schema file. The nextflow schema file helps you as a user. To understand which parameters are available in the pipeline and if you have added any wrong parameter in the pipeline. Through the schema, you can help the NFCO tooling. You can see UMI Options, Skip UMI Extract, UMI BC Pattern etc. When you go to the NFCore website's pipeline specific page and click on the parameters, you will find the same options here. And this page is generated with that file. You can use the nextflow schema.json file to render this dynamic form. Now, why have we introduced this? Because whenever you want to use any pipeline, the most important aspect of NFCore as a user is how you can change the parameters of the pipeline. The second is how you can change the configurations. Now, we will talk about the parameters of the pipeline. Now, we will go back to our Gitpod environment. And here we see that if you want to launch the pipeline and if you want, you can see the web-based setup. So, the web-based setup doesn't look so good in a terminal. So, I will exit from here and go. In this URL, which the tools have already generated for us. And as soon as I come to this URL, I can see the parameters of that pipeline and I can customize it here myself. Here, I will say, Rnseq, we say, NFCore tools run one. Whatever you want to name here, if I say, test docker or some custom work directory, generally you don't want to customize the work directory but for now, I am changing it. And if you want to resume this, how can you do it? Now, if you launch, along with this, before I tell you about the launch, if you scroll down, you will see all the options which are available in the parameters page of that pipeline. Now, we go back to that link which NFCore tools have generated for us. Now, here we can customize those parameters. As soon as this is true, here I am doing true, here I am doing false. And by doing this, I can customize any option here which is correct for your environment. So, it is possible that your work directory location is different. But once you use NFCore tools and want to set the parameters of that pipeline, then you can get this visual setup. And when you click on launch, then you get an important feature. When you click on launch, there will be validation as well. And as soon as there is validation, you will get to know that your output is not correct. I can give you this. The parameters which are not output, are all validated or not. Now, if we click on launch, then all the parameters are validated again. After that, because we have an internet connection, we will directly use this command. And if your system does not have an internet connection, then you can use this command directly. But now, we will take advantage of the internet connection. Along with this, you can come back to your Gitpod environment and paste this command and run it. Now, here you can see that NFCore tools, which you have edited in that link, you have picked it up, you have converted it to a JSON file. And if we want to see that JSON file in our explorer, then we can see here, JSON. I will reduce this window a little bit to the terminal so that we can see the JSON file. We have only changed these things. And the other things that we have changed, like name, etc., they have been accommodated in the next flow command. This is also good for the profile, because the profile parameter has also been accommodated. Now, if I say yes, then we have started the same workflow with the layer of NFCore. So, if you use the baseline next flow tool, then you can run the pipeline. But you don't get the validation of the schema. You will get it only after the pipeline starts. When you are doing the results analysis, or your pipeline is half completed, then you will get the validation of it. So, you can use the NFCore tools or the launch option with your downloaded pipelines. And as we have already talked about, you can use NFCore tools. to launch any NFCore pipeline. Along with this, we will see that we have covered the launch, we have covered the downloading pipelines, how we can download the assets of a pipeline, like Singularity, etc., we have covered it too. So, we have covered most of the usage-oriented aspects. So, one thing that remains is our configuration files. So, now we will talk about that. After a short break. Now, we will talk about the configuration of the next flow or NFCore pipeline. We will talk about it in more in-depth tomorrow, in the next sessions. But for now, we want to show you that if you want to use NFCore pipeline in your infrastructure, then how you can use the custom configuration. So, you may have noticed that the NFCore RNA-seq pipeline was locally downloaded. If we go inside that folder, then we will see that there are three folders, configs, Singularity Images, Workflow. We were using the Workflow folder to launch NFCore tools and to use local launch functionality. So, if we launch NFCore, then we have the option to tell NFCore tools to remotely pull a remote pipeline or to use a pre-downloaded pipeline. So, we have seen the use case of launching a pre-downloaded pipeline. Now, we will see how we can customize it. For hardware, the main bioinformatics logic we will let it be intact. We will not change that. Now, we will do the CD configs. Here, we can see that there are many files. And if I do tree here, then we will install tree tree. So, we can see that there are many files available here. But we are not interested in MD files, we are interested in .config file. So, we can add our custom configuration file here. For example, if you go to CDconfig and here, you can see that there are many configuration files here. But all these configuration files the members of the institutes have contributed to NFCore. For example, someone has contributed to the configuration of Azure IWS batch. Someone has contributed to the configuration of Azure batch. Similarly, people make a common configuration file for their institutes which they can contribute to NFCore. So, if you are interested in this, definitely the next video, our next module is interesting for you. It is important for you because in the next module, we will talk about how you can use the NFCore developer and development tooling. For now, we will go back to the top level and open this nextflow.config file. Or we can even open the nextflow NFCore.custom file. By using the explorer, we can see the NFCore pipeline, config. You can see the NFCore custom here. So, we can see here all the individual configuration files are included here as a profile. But if you want to add your own configuration, you can simply create a profile. For example, I can give the name of my profile. Right now, I want to customize something in the paramscope. So, I am giving the name. OutDIR. I want to customize OutDIR parameter. For this, I will give the name. RNaseek NFCore Results As soon as I save this file, because I have added this in the custom configuration and if we look at the nextflow.config file, this file has been imported there. So, this means that in the nextflow.config file, I have added a profile of my name. And now, we will see how we can use it to run this RNaseek pipeline. Maybe Yes. Now, we will run that pipeline again. NFCore Launch. Again, local path. Now, we will call workspace, tools, training tools There is an s in it. Workspace Tools With it, training nfcore and in it, workflow and with it, we can enter. Now, on the basis of command line, if you want to customize your parameters, you can see that what I want to give the name. Now, custom name. Now, the value of the custom name is set. I can give the profile. Test Docker Right now. And if I click on continue, then I can see that Out TIR is a required parameter. Out TIR, I will give the value here. Custom Out TIR I entered and with it, I can see that there is no other required parameter. So, I will continue. And I have not customized anything else. With it, a new parameter file was generated. If you come back and see in your explorer. So, in this folder, there are many files. So, maybe we can introduce a trick. If you open the command palette in VS Code, then you can give the name of the file directly. nfparams and you can open the file directly from here. This file is not made on the other place. Because we gave the name of Out TIR . So, here it is. So, that is the second file. nfparams, which is in the folder of downloaded pipeline. The first file we were using, we were not using it. So, maybe we can delete it so that there is no confusion. This time, we run it. Now you can see that like, I move my head short that this time, test docker, we had mentioned the profile. So, nfco tooling has told the next flow that we have to use this profile. Now, the next flow is launching that pipeline. Along with this, I would like to show you that you have infrastructure related how many configuration settings are available. So, if you search next flow configuration, then your first link should be taken on the website. This one. So, here you can see that next flow actually has a lot of scopes in the configuration file. We have only talked about one type of scope which is parameter scope. But there are many scopes available. So, you can see that this is a hierarchy configuration sources are reported below, listed in order of priority. So, priority order which is followed every time to know which configuration you want to use. This is the syntax like you might remember from yesterday's video from yesterday's session. So, how do we include the config file? We have seen this too today. And now, we will talk about the scopes as soon as we have seen parameter scope. So, parameter of the file in our command I will cancel this command because you can see that this execution started but I mainly want to show you I want to show you I have to scroll a little up. No, it is not possible yet. So, I want to show you params. So, this nfparams.json file is a hyphenparams value of hyphenparams argument for the next flow value. So, by doing this you can provide custom parameters but with that if you want to customize any nfcore pipeline or customize any next flow pipeline then you can use the executor scope. Along with that, you have a process scope you might remember that we used container directive or conda directive and directives you will get on the process page you will get a full list there. Along with that, you can experiment with many things like reporting functionality of next flow or tower functionality or trace functionality which can help you to do a particular execution with hardware usage and software usage you can provide an overview. So, on the process page you can see how many directives are available you can open this menu and you can see how many directives are available. Along with that our nfcore tools introduction from the user point of view is over. In the next section we will talk about how you can use developer nfcore tools. As we have taken the user overview of nfcore tools we will take the developer overview and we can see that nfcore tools are available on the official website within the talks section creating pipelines with nfcore. In this, we can start creating a pipeline because this is our githpod environment setup we have introduced nfcore tooling along with that we can start working with nfcore create. Now, first of all I will make another folder developer training and in this I completely reload the githpod so that we can create new files in that folder and whatever we have created in the current location we have no interference with them so for that I will open folder again and tools and this time developer training along with that the website will be reloaded but related to nfcore tools whatever information you need you can take it out from our tools tab the tools tab on the website all the information about tools everything is mentioned here as you can see the tools on this website all the options are available and all the settings are available along with that there is another linting specific configuration of nfcore tools which you can see by searching a yaml file we will use this but first we try to create a pipeline we come back to githpod we say yes I will close the ports and developer training folder and we will start our discussion along with that we are in developer training folder let's see the help of nfcore tools and close the explorer so developer related we have some commands like create, lint, modules subworkflows, schema bumpversion and sync so we will discuss these things and we will start nfcore create if I type nfcore create command on my terminal so nfcore tools will give me an interactive menu through which I can apply whatever I want to achieve like I can give this workflow devflow devflow for now we give this name description random description author abhinak sharma maybe abhinak sharma now we don't want to customize anything so we press capital N and along with that nfcore tools we have made a folder nfcore devflow name so this is our test pipeline we will do linting changes and along with that we will introduce some features of nfcore tools so if you pay attention first of all nfcore tools gave you feedback that it has made a pipeline it has initiated a gate repository but now it is your responsibility to set up a remote of that pipeline so this as soon as you set up a remote after that after that after that the sync command or the collaboration with whom you are doing you can do it easily if you share this code on github or any version management hosting solution along with this we will go into nfcore devflow and we will see which files we have maybe you remember we saw nfcore full scale rna seek pipeline on github so it had similar structure here ben,conf docs modules,subworkflows and other files are linting automation tests and automated we can say github actions are being used overall many automations are being used for different aspects but as you can see github nextflow.config file which we are already familiar with but along with that .nfcore yaml file which i have mentioned along with that is docs folder which has some markdown files and some images this docs folder is used to generate documentation on github so all these files are used either in quality control or in documentation along with that we can see if we come into this pipeline folder then we can use lint command because as soon as the pipeline is built if you do not change there will be some errors but they are by default there will be some errors but they are by default because there are many to-dos in this template and the linting rules i hope you are addressing to-dos now let's see by default what will be the result you can see that our zero test failed there are 79 tests passed and there are 23 warnings this means that to-dos are not the best practice but this is just a warning because this is not an error this means that you can use it along with that i want to highlight that two warnings are from specific modules their reason is that they are available we will talk in a little while but for now we will talk about linting if i do a change-institutes in a mark-down file like code of conduct this file gives your contributors a kind of guidance that if someone wants to contribute then how can he do it and what is the correct and expected behavior within the NFCore community if i for example i don't follow these rules or i try to break this mark-down this is wrong or i try to break another mark-down citations, change log if the readme of the pipeline the readme of the documentation raise warning so when you do linting linter will give you feedback about these changes now you can see here that two files are failing because we haven't followed the mark-down rules and second there are some files in which you don't need to change like code of conduct and community-level code of conduct so all the pipelines are the same all the pipelines code of conduct is the same which is a copy of template code of conduct so you might remember that NFCore's all the tooling is a template around so that the best practices are implemented in one place and the pipelines that are made can automatically include because they have been included in the template along with this i undo these changes so that the linter can be happy again and if i come back and restore the code of conduct then our linter will come back in the state before and along with this we can see if i run the linting again that everything is going well or not apart from that we have other kinds of linter like prityer's linter python's linter if you want to run prityer's linter then you can see prityer-c is to check these changes won't be introduced but if you change again and run prityer then you can see that prityer will give you feedback in this case prityer didn't give feedback because this file this file doesn't cover prityer prityer covers jason so if we try to change jason then we can see that prityer will give overall feedback prityer has given us feedback that the modules and jason file's formatting is not as expected and this formatting the template of nfcore is expected that there are spaces after any scope there are spaces in key value pair it's not an honour so you can use prityer to fix this change prityer-w so prityer will see what should prityer do so that the file doesn't comply so here as you can see prityer has automatically updated and these files comply similarly you can see that in bin folder maybe you remember the importance of bin folder which we covered in the end of simple rna seek pipeline in which we put the custom script I want to cross it and move it let me move it in which we put the custom script fastqc.sh similarly you can put any language's custom script in this case we give nfcore default script which is checksamplesheet.py if you introduce a change which doesn't match linter's expectation for python which we use is called black if we type black. if we type so it tells that whatever we have changed there is no problem with this linter but if we some best practices go against like functions or methods or new line we remove if we run black again we can see this time we have reformatted a file and the empty spaces and new lines are restored again you can see there are many linting techniques which come with nfcore template and you can use it so that you can easily comply with your best practices nfcore templates along with this to cover nextflow schema nextflow schema you might remember when we saw nfcore rnseq pipeline here when we saw nextflow schema then we talked about how this schema the parameters of that pipeline are used to render the menu now we will talk about how you can build and lint because the schema is very important to help your users and first of all for the usability of your pipeline now we will see how we go inside the gitpod and return to see how we can build first nfcore schema first let's see which options are available there are 4 commands available we want to do linting so we will type lint and pipeline schema we forgot to dot because this command we can use local schema with a remote pipeline but we want to use with local pipeline so we added dot and our schema is valid our linter is happy after that if we introduce a change in the config file then we can see what can be used when we introduce a change in the configuration without changing the schema so now we will add 2 or 1 parameter temp parameter we can name it xyz you can name it delete me you can name anything hopefully your name will be relevant to your pipeline I am just a demo pipeline for this I have given random names and if you first run this lint after that we will come back to the schema here we don't want dot here we can see a test is failing now we will see why it is failing parameter temp parameter from nextflow.config not found in nextflow schema this means our linting test your nextflow config file in that you mentioned the parameters after that you mentioned parameters in your nextflow schema there is a mismatch so this linting can help you directly in this if you test nfcore schema and this time we will use build now you will use nfcore tooling yes using nfcore tooling you can resolve mismatch nfcore tool directly asked me if you want to add parameter temp add web builder now we can test how web builder will be if we open our browser tab is open as we saw nfcore launch parameter schema editor interface now using this view you can design a special nextflow schema here you can decide if parameter is required or not parameter is required validation pattern format options 3 or 4 options must be evaluated and you can set these categories all these things you can use nextflow schema file and use nfcore tools you can easily build now you copy and save on your editor if you open nextflow schema file we will use a shortcut and search nextflow schema I can update this file here we can see our new parameter type string default value delete me but if I want again I can open nfcore schema editor for that we have to type nfcore schema build and use it as we use we have nfcore tools we want to start web builder if we say yes we have the same option we can enhance as you can see our custom parameter is coming here we can give it a description this is the sample description we can change the type integer boolean string here we can say it is a required parameter if we want to give a specific logo we can give that too click here if I want we have given this icon and when we have done all the edits we can finish by clicking we can copy the entire schema and go back to our editor we can replace this schema because our schema is updated required temp parameter and we have an icon although nextflowschema.json file is quite big but the nfcore tooling for this schema file can help you a lot to use it in your pipeline so that your pipeline is user friendly and can give meaningful feedback before launching the pipeline after launching the pipeline after that now we want to talk quickly we have talked about linting about continuous integration if you build your pipeline with github then whatever actions we have built .github in actions folder our github is reloading as soon as .github folder opens you can see issue templates and workflows as soon as you open the workflows you can see every stable release there are many test triggers or if you want to merge then every pull request is done through rigorous testing like linting ci aws test whatever pipeline is stable release in some conditions stable release comes as soon as this condition is satisfied these workflows are triggered and the pipeline is tested in infrastructure now we can talk about linting to customize some cases where you especially development time when you don't want to enable linting .nf-core.yaml you can use this and you can add options whose examples are on our tools website like actions, aws full test falls pipeline to do's, falls if we add this in our pipeline in our setup as soon as we follow that format we can see how we control linting behavior now we come back to our terminal my terminal it was a little freezed but now it is back now we can try nf-core.lint as soon as we run lint this time our warnings will not be displayed you can see only 3 warnings which are module's origin is related and schema is related to description because we did not add any description separately by doing this we can control behavior along with this now we are ready to move towards modules how you can create a module or how you can download modules along with this we will transition to next module as we have talked about schema creation we will talk about modules and sub-workflows so modules maybe you remember that we have already mentioned modules nf-core is a sub-project in which you can download many modules which are readily available formatted and reusable from nf-core website you can download them or update them so now in nf-core modules project there are 832 modules which you can download and use you will get input output in this module listing page its idea but if you open a module click on the page click on the module link it will open a module specific page where you will get relevant information so assume that we want to install this module in our pipeline so we will copy this command and you can see what is the description what is the input what is the pattern what is the publication what is the license what is the relevant documentation all this by default and you can get modules in sub-project now if we want to install this module in our pipeline we just need to install nf-core and whatever change will be implemented on left hand side you can see in modules folder now I want to tell you one thing because these modules there are 2 sub-folders like local and nf-core using nf-core tools we want to download a standard module so it will be in the nf-core folder but if you want to make a custom module for your pipeline which may not be present in nf-core it will be in the local folder so lets start according to me its name is only 1s perfect now lets install this tool and if we see our module.json file now githpod will show us that there is a small change module.json file this is the structure of module.json till now by default the template is that structure but as soon as we installed this file is updated so this file updates nf-core tools so that it can track which module which version which version is downloaded now this is not the version of the tool this is the version of nf-core module which uses the version of a tool so you can control these things through configuration which we will talk about later now we have seen how to install this module if you want to remove the module we will see how to do that we can give the full name or we can type nf-core module remove and we will get interactive option which gives nf-core tools so that we don't have to search now we will see which and then we just press and now you can see module.json the entries of abacus have disappeared and our file system abacus has disappeared which was before now if we want to install another tool like first we will see that local I will increase it first so that we can see it now if we want the currently modules installed see a list we can give this command nf-core modules list if we enter here we will get more information that it has two commands local and remote first we will see which modules are installed after that we will see on remote this command nf-core tools this module.json file and you can tell which modules are installed which repository what is the version of this module not of this tool a message and date but if you want to see in remote how many modules are available you will get this entire list this 832 modules list and maybe you want to see this entire list if you want to see some of these things let's see what is the option in help here is jason output if you want to filter these modules you don't need to go to the website you can use jason output to filter or directly use this command to filter like samtools there are so many sub modules available for samtools samtools tool is a tool name and option name this is design pattern for nf-core modules you will write the name of the tool after that we will write the name of the option later let's try to install samtools view nf-core modules install if i don't see the module name and run this command then nf-core tools will give me interactive menu in which i will type samtools name it will give me auto-completion when we want to install the view then we type and press the tap button after that you can see that the first thing our module.json file is updated again in which now samtools view is included second samtools view is installed in our nf-core modules folder now another thing nf-core tools tells us that our samtools view is added in one file if you want to use the following statement to include this module if you want to include this module in your main.nf workflow or in sub-workflow then you can give this command i can show you an example if you want to include main.nf then go to the top not main.nf maybe i will tell you the structure also so when you open main.nf you can see in this file the first thing is dsl2 is enabled this is common for everyone after that it gains parameter values and then print here we do validation after that you import a workflow and invoke this workflow now if you see in this workflow you will see the defflow file here all modules are imported and modules are being used so we can import these modules and because we downloaded this module in nf-core we can add formatting with this now we have added this module in our workflow file similarly if we want we can install any modules and we can include them in our workflows but if you want to see what are the available parameters you don't need to go to the website which you can do but for you if you are used to command line then this option will be easy for you and samtools viewed from here you can directly print the information of the module whether it is a local module or a remote module you can compare nf-core tools and you can decide from here how to design the input and output with this we see more commands which are covered in nf-core modules now you can see some commands are on the level of pipelines and some are related to new module development we will talk on the screen level and we will see we have covered the information of a module we will see how to patch a module so we believe that if you want to customize any nf-core module then we will see we will open samtools view and we will open main.nf file main.nf file you should look familiar because there is only one process which we discussed yesterday nf-core modules sub-project it uses the stability of the next flow and builds it you can see there is only one process and we have used stubs this is a little advanced usage but the meaning of stubs is that you are telling the next flow that instead of executing the script section you can create dummy files so that you can check the module logic along with this you can see that these are all well formatted so whatever linting rules we have seen on pipeline level same linting rules now we want to change a pattern to see what will be the effect of this and if we run nf-core lint we will see that linting tool is not our because we have changed a nf-core module and we have now this pipeline one way reproducible step or I would like to say that nf-core checks and balances with development doesn't comply with them means you are not following the best practices and some things are not obvious for nf-core tools so if I want to change this module I am completely sure I will use nf-core modules patch command now if we use patch command we can see view samtools view index out of range this means that we made something wrong if we notice here this name is not complete this means if we run and ensure that the name is complete we type view and this time patch command has made a correct pickup and we have information changes in module changes in this file after that information is meta yaml is unchanged and patch file is saved in a diff file and our diff is this now let's see the diff file which we can see on the left hand side if we open the diff file these are our changes vscode or gitpod you can see in this format so we can see that we used local change instead of using local change now if we see modules.json file samtools view entry this means now nfco tool is aware of changes it knows that these are changes and it won't complain on time because we followed the best practice to update nfco tool that our local change is please track it after this we see by lint this time our linting error should not come and zero linting errors but now we can address these things pipeline test warning error warning message new version is available now we use another functionality update if we update let's see named module update and all modules because we have 2 warnings test pipeline all modules update just update everything ok with this all modules are updated now you can see this line is changed now the module is fetched from a particular revision and that revision is changed we can only change one module but in this case we have decided to update so with this now we move towards local modules how we can create local modules and what is the use case of local modules first of all the use case of local modules is that if you nf core whatever module you want is not present in nf core so this means you can't use nf core tools to manage that module to create so let's create modules create and we name that module dev module so you can see we have a warning nf core tools naconda api where bioconda packages are published confirm that there is a project or a tool available and it gave us feedback if we can stay here or we stay here and create a new module for that tool we know that it is available like sam tools or bwa mem this module is available in nf core tools but as an example I will try to make this module again so that I can show how you can make this module bioconda package nf core tools identified so you can customize the package version later nf core tools docker and singularity version confirm that this bwk version docker and singularity version as soon as this process is completed we can see that our local folder has been created within it I made it a little fast forward so that the internet delay can be overcome and we can move forward with our project now biocontainers we did not get a definite version but it may be a temporary issue in that api normally through that api docker container version and singularity container version and address today I give the author's name I am Abhinav if I want I can select some medium or high process with this we require metamap metamap is a design pattern in modules means you can send extra information regarding a particular sample in metamap for now we say we do not need it and nf core tools have been made locally modules in local folder bwa project has been created now let's see if this project track is in modules.json or not you can see that local module is not tracked that means whatever changes will be in this module they will only enter in your version control tracking not in the tracking of nf core tools now you may have seen so many to do you do not need to understand all this you can delete all this and just pay attention which is essential for your module design you do not need to panic all this is part of best practices so if not today then tomorrow you may need it but for now you can continue so by doing this you can create a local module and if we want to see which other commands are available for modules create test yaml command lint command so let's lint and see our modules so we have two warnings which are bioconda version has been updated and the name of the process is not used in this module apart from this we do not have any separate issues most of our tests are passed and if you want then for your module I will run one more module so that I can see and if you want to see create test yaml then you can see pewa let's see this is only used for development of nf core modules so now you can't see our local module but if you want to contribute in this project if you are interested in using this project or adding new modules then you can open the modules project and see its best practices and documentation which is available on the website and you can see this project you can see the modules which modules are available and how they are designed and automated tests so all this information if you want to contribute in modules then you are visible here along with this now we will transition to sub-workflows with modules experience and all the sub-workflows available on nf core are also in this modules project so what you can do if you go back to documentation of tools then you can go to its introduction or table of contents and see which modules are available and each command documentation app you can do that and you can interact with sub-workflows like if you want to see in our pipeline nf core sub-workflows list which sub-workflows are available then you can see in remote we have these common sub-workflows like if you want to install BAM and sam-tools common combination with that BAM starts sam-tools if you want to install those sub-workflows then you will use similar command nf core sub-workflows install BAM starts sam-tools and nf core tools will install those sub-workflows in your sub-workflows folder as you can see here same modules folder like a local sub-workflows folder nf core sub-workflows folder you can see here that BAM starts sam-tools sub-workflows have been set up here now we will see module.json file so that we can correlate what have been changed in this file you can see here in sub-workflows section new sub-workflows have been added along with this you can see here sam-tools idx flak-stats these sub-workflows which are used are also installed in your modules directory nf core tools so a sub-workflow is made of many modules and the modules are definitely made of tools but the sub-workflows are 3 or 4 modules you can depend on any n number of modules and when you install sub-workflows nf core tools you can install those sub-workflows dependency modules so that you don't have to manage along with this I would like to tell one more thing that if you want to integrate sub-workflows in your main pipeline or workflow then you can use the name of the pipeline to include it as I have included here along with this you can run the linting process so that you can see that everything is being integrated or not and along with this I would like to say that our session 2 concludes and I hope that this session was useful for you and you can participate with nf core community and contribute if you are a member of this community then it would be very happy for us and the more the members are the more the testing of the pipelines and the quality check of the pipelines and if you use these pipelines in your analysis or in your collaborations then please site these pipelines and nf core community because these citations are very important for the nf core community and it is very important to connect so that nf core community can get funding like nf core community it is still funding from Chand Zakhar book foundation and many cloud providers but it depends on the momentum of the community so as you can see as we go to the end of this page supported by Chand Zakhar book foundation microsoft azure and aws the more the community we have the more the momentum we can help people because open science and open community and open source this can be sustainable only when our community is strong so thank you so much for joining we hope that you will talk to us on nf core Slack and if you have any questions then please contact us on our channel channel name will be 23 mar mar training hindi as soon as you search you will get this channel so thanks for today we will continue tomorrow with session 3 bye