Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
dstools [2023/04/06 10:30] – Didier Barradas Bautistadstools [2024/11/21 07:22] (current) – Didier Barradas Bautista
Line 5: Line 5:
 ===== Overview ===== ===== Overview =====
  
-Data science is a field that combines domain expertise, programming skills, and knowledge of mathematics and statistics to extract meaningful insights from data. It is a rapidly growing field with a wide range of applications, including business intelligence, healthcare, finance, and marketing.+Data science is a field that combines domain expertise, programming skills, and knowledge of mathematics and statistics to extract meaningful insights from data. It is a rapidly growing field with many applications, including business intelligence, healthcare, finance, and marketing.
  
-The KAUST Visualization Core Lab (KVL) is hosting several hands-on workshops as part of ongoing efforts to build capacity in core data science skills at KAUST and in the Kingdom. We try to make all the material available on several channels like the ones you will find below+The KAUST Visualization Core Lab (KVL) is hosting several hands-on workshops as part of ongoing efforts to build capacity in core data science skills at KAUST and in the Kingdom. We try to make all the material available on several channels, like the ones below.
    
  
 ===== GitHub ===== ===== GitHub =====
  
-Our [[https://github.com/kaust-vislab|GitHub]] repository contains code and data for various data science projects. For more material, you can also check the Kaust RCCL [[https://github.com/kaust-rccl|GitHub]].+Our [[https://github.com/kaust-vislab|GitHub]] repository contains code and data for various data science projects. For more material, check the Kaust RCCL [[https://github.com/kaust-rccl|GitHub]].
  
 +/* 
 ===== Confluence ===== ===== Confluence =====
  
Line 20: Line 21:
 ===== YouTube Videos ===== ===== YouTube Videos =====
  
-We have made several YouTube videos about data science. You can them [[https://www.youtube.com/@kaustvisualizationcorelab4612/videos|here]].+We have made several YouTube videos about data science. You can them [[https://www.youtube.com/channel/UCR1RFwgvADo5CutK0LnZRrw|here]]. 
 +*/
  
 ===== Data Science for Beginners ===== ===== Data Science for Beginners =====
  
-This [[https://youtu.be/Pcdsz6uKh8g|video]] provides an overview of how to use the HPC clusters to start your data science projects. And this video will guide you to [[https://youtu.be/QCt8UHK5bPo|setup]] your Windows machine for data science.+This [[https://youtu.be/Pcdsz6uKh8g|video]] overviews how to use the HPC clusters to start your data science projects. And this video will guide you to [[https://youtu.be/QCt8UHK5bPo|setup]] your Windows machine for data science.
  
 ===== Data Science Tools and Technologies ===== ===== Data Science Tools and Technologies =====
Line 34: Line 36:
   * [[http://myws.kaust.edu.sa/|Remote workstations]] (you have to request the service the first time you use it [[https://kaustforms.formstack.com/forms/remote_workstation_account|here]])   * [[http://myws.kaust.edu.sa/|Remote workstations]] (you have to request the service the first time you use it [[https://kaustforms.formstack.com/forms/remote_workstation_account|here]])
  
-===== Data Science Workflow ===== 
  
-This video walks through the data science workflow, from data collection to data analysis to model building.+/* 
 +===== Data Science-specific information on Ibex ===== 
 + 
 + 
 +There are further materials of interest for a Data scientist who is going to use Ibex like the [[https://www.hpc.kaust.edu.sa/ibex/best-practices-deep-learning|best practices for Deep Learning]], [[https://www.hpc.kaust.edu.sa/ibex/best-practices-using-conda-pip-ibex|Best Practices for Using Conda (+Pip) on Ibex]] and [[https://www.hpc.kaust.edu.sa/ibex/containers|containers for HPC clusters]]  
 +*/ 
 + 
 + 
 + 
 +</WRAP> 
 +<WRAP center round box> 
 +===== Ibex Starting Pack ===== 
 + 
 +<WRAP center round box important >  
 +===== Access to ibex ===== 
 +Unless your username is associated with either a Project, Principal Investigator, or a Class, access to ibex resources will be [[https://docs.hpc.kaust.edu.sa/policy/ibex.html#limits-on-unassociated-users|severely constrained]]. To nominate your supervisor, please visit http://my.ibex.kaust.edu.sa . You can check if a PI is defined using the "whoismypi" program on a login node. 
 + 
 +To use Ibex resources in any significant way, you must be under the direct supervision of the **Research Faculty or a Director**. From RCAC clarification, it is only professors (full, associate, assistant, and research). 
 + 
 +</WRAP> 
 + 
 +Welcome to KAUST! As a new user, especially if it's your first time, here are some essential resources to get you started: 
 + 
 +1. **Ibex 101**: Begin with the [[https://docs.hpc.kaust.edu.sa/training/ibex_user_resources.html#ibex-user-resources|Ibex 101]] slides and demos. These will introduce you to all the resources and commands available in the cluster. This foundational knowledge will be referenced frequently. 
 + 
 +2. **Ibex Training**: Explore our detailed documentation on various topics, including Data Science, for more comprehensive information. Be sure to check out these wikis: 
 +   - [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/index.html|Ibex User Guide]] 
 +   - [[https://docs.hpc.kaust.edu.sa/systems/ibex/index.html|Ibex main docs]] 
 +   - [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/datascience.html#data-science-on-ibex|Data Science User Guide]] 
 + 
 +3. **KAUST Visualization YouTube Channel**: Visit our YouTube channel [[https://www.youtube.com/channel/UCR1RFwgvADo5CutK0LnZRrw|here]] for many tutorials on data science workshops and more. These videos are beneficial, covering the best practices for installing libraries and coding with Ibex. 
 + 
 +4. **Python and Conda**: on the subject of managing your environments, we encourage you to install Conda using the official guide [[https://github.com/kaust-rccl/ibex-miniconda-install/tree/master|here]] 
 + 
 +5. **Jupyter, VsCode and Rstudio**: To deploy your prefered IDE on ibex check the video [[https://youtu.be/88pZEvKAiHY?si=010dJPGtGQQwMu8W|here]] 
 + 
 +<WRAP nicebox yellow 98%>  
 +===== Important storage tips ===== 
 +  * **/home/$USER** - this is your personal home directory and is limited to 200GB of data. ** Do not run HPC jobs in your home directory ** - use it to keep configurations, scripts, etc. 
 +  * **/ibex/user/$USER** - this is your HPC storage. Use this storage to save the output and input to compute jobs. **This is much faster than your home directory**, so your jobs will run faster. This directory has a limit of 1.5 TB. If you need more storage, contact the Ibex support team. 
 + 
 +</WRAP> 
 + 
 + 
 + 
 + 
 +These resources will equip you with the knowledge and skills to maximize your time at KAUST. We hope this docuwiki page is helpful! Please let us know if you have any questions. Happy learning!
  
-We hope this docuwiki page is helpful! Please let us know if you have any questions. 
  
 </WRAP> </WRAP>
Line 44: Line 90:
 ---- ----
 <WRAP group> <WRAP group>
-<WRAP center round box important half column > ** Further useful links** +<WRAP center round box half column > ** Further useful links**
- +
-  * [[https://www.hpc.kaust.edu.sa/ibex/ibex_guide|Ibex user guide ]] +
-  * [[https://kaust-supercomputing-lab.atlassian.net/l/cp/nM0wqF7a|Ibex Starting pack]] +
-  * [[https://www.hpc.kaust.edu.sa/user_guide|Shaheen II user guide]]+
  
 +  * [[https://docs.hpc.kaust.edu.sa/index.html| KSL Support Documentation ]]
 +  * **ibex@hpc.kaust.edu.sa** - email this address to create a service request
 +  * https://kaust-ibex.slack.com - Use **#general** for simple queries
 </WRAP> </WRAP>
  
-<WRAP center round box important half column > {{:icon:twbs:link:w-25:h-auto:headset-vr.svg?nolink&}}** KVL Training on Data Science** 
  
-  * KVL often runs training on Data Science in collaboration with the KSL. Check out our current and past workshops for more: <wrap button>[[:training|Training Events]]</wrap>+ 
 + 
 + 
 +<WRAP center round box half column > {{:icon:twbs:link:w-25:h-auto:headset-vr.svg?nolink&}}** KVL Training on Data Science**  
 + 
 +  * KVL often runs training on Data Science in collaboration with the KSL. Check out our current and past workshops for more: <wrap button>[[training:overview|Training Events]]</wrap>
  
 </WRAP> </WRAP>
  
 +</WRAP>