Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
dstools [2024/08/20 09:52] – Didier Barradas Bautistadstools [2024/11/21 07:22] (current) – Didier Barradas Bautista
Line 18: Line 18:
  
 Our [[https://kaust-supercomputing-lab.atlassian.net/l/cp/1JZzoASv|Confluence]] page contains recipes and other resources for data scientists. Our [[https://kaust-supercomputing-lab.atlassian.net/l/cp/1JZzoASv|Confluence]] page contains recipes and other resources for data scientists.
-*/ 
  
 ===== YouTube Videos ===== ===== YouTube Videos =====
  
 We have made several YouTube videos about data science. You can them [[https://www.youtube.com/channel/UCR1RFwgvADo5CutK0LnZRrw|here]]. We have made several YouTube videos about data science. You can them [[https://www.youtube.com/channel/UCR1RFwgvADo5CutK0LnZRrw|here]].
 +*/
  
 ===== Data Science for Beginners ===== ===== Data Science for Beginners =====
Line 36: Line 36:
   * [[http://myws.kaust.edu.sa/|Remote workstations]] (you have to request the service the first time you use it [[https://kaustforms.formstack.com/forms/remote_workstation_account|here]])   * [[http://myws.kaust.edu.sa/|Remote workstations]] (you have to request the service the first time you use it [[https://kaustforms.formstack.com/forms/remote_workstation_account|here]])
  
-===== Data Science-specific information on Ibex ===== 
  
 /* /*
 +===== Data Science-specific information on Ibex =====
 +
 +
 There are further materials of interest for a Data scientist who is going to use Ibex like the [[https://www.hpc.kaust.edu.sa/ibex/best-practices-deep-learning|best practices for Deep Learning]], [[https://www.hpc.kaust.edu.sa/ibex/best-practices-using-conda-pip-ibex|Best Practices for Using Conda (+Pip) on Ibex]] and [[https://www.hpc.kaust.edu.sa/ibex/containers|containers for HPC clusters]]  There are further materials of interest for a Data scientist who is going to use Ibex like the [[https://www.hpc.kaust.edu.sa/ibex/best-practices-deep-learning|best practices for Deep Learning]], [[https://www.hpc.kaust.edu.sa/ibex/best-practices-using-conda-pip-ibex|Best Practices for Using Conda (+Pip) on Ibex]] and [[https://www.hpc.kaust.edu.sa/ibex/containers|containers for HPC clusters]] 
 */ */
  
  
-We hope this docuwiki page is helpful! Please let us know if you have any questions.+ 
 +</WRAP> 
 +<WRAP center round box> 
 +===== Ibex Starting Pack ===== 
 + 
 +<WRAP center round box important >  
 +===== Access to ibex ===== 
 +Unless your username is associated with either a Project, Principal Investigator, or a Class, access to ibex resources will be [[https://docs.hpc.kaust.edu.sa/policy/ibex.html#limits-on-unassociated-users|severely constrained]]. To nominate your supervisor, please visit http://my.ibex.kaust.edu.sa . You can check if a PI is defined using the "whoismypi" program on a login node. 
 + 
 +To use Ibex resources in any significant way, you must be under the direct supervision of the **Research Faculty or a Director**. From RCAC clarification, it is only professors (full, associate, assistant, and research). 
 + 
 +</WRAP> 
 + 
 +Welcome to KAUST! As a new user, especially if it's your first time, here are some essential resources to get you started: 
 + 
 +1. **Ibex 101**: Begin with the [[https://docs.hpc.kaust.edu.sa/training/ibex_user_resources.html#ibex-user-resources|Ibex 101]] slides and demos. These will introduce you to all the resources and commands available in the cluster. This foundational knowledge will be referenced frequently. 
 + 
 +2. **Ibex Training**: Explore our detailed documentation on various topics, including Data Science, for more comprehensive information. Be sure to check out these wikis: 
 +   - [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/index.html|Ibex User Guide]] 
 +   - [[https://docs.hpc.kaust.edu.sa/systems/ibex/index.html|Ibex main docs]] 
 +   - [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/datascience.html#data-science-on-ibex|Data Science User Guide]] 
 + 
 +3. **KAUST Visualization YouTube Channel**: Visit our YouTube channel [[https://www.youtube.com/channel/UCR1RFwgvADo5CutK0LnZRrw|here]] for many tutorials on data science workshops and more. These videos are beneficial, covering the best practices for installing libraries and coding with Ibex. 
 + 
 +4. **Python and Conda**: on the subject of managing your environments, we encourage you to install Conda using the official guide [[https://github.com/kaust-rccl/ibex-miniconda-install/tree/master|here]] 
 + 
 +5. **Jupyter, VsCode and Rstudio**: To deploy your prefered IDE on ibex check the video [[https://youtu.be/88pZEvKAiHY?si=010dJPGtGQQwMu8W|here]] 
 + 
 +<WRAP nicebox yellow 98%>  
 +===== Important storage tips ===== 
 +  * **/home/$USER** - this is your personal home directory and is limited to 200GB of data. ** Do not run HPC jobs in your home directory ** - use it to keep configurations, scripts, etc. 
 +  * **/ibex/user/$USER** - this is your HPC storage. Use this storage to save the output and input to compute jobs. **This is much faster than your home directory**, so your jobs will run faster. This directory has a limit of 1.5 TB. If you need more storage, contact the Ibex support team. 
 + 
 +</WRAP> 
 + 
 + 
 + 
 + 
 +These resources will equip you with the knowledge and skills to maximize your time at KAUST. We hope this docuwiki page is helpful! Please let us know if you have any questions. Happy learning! 
  
 </WRAP> </WRAP>
Line 49: Line 90:
 ---- ----
 <WRAP group> <WRAP group>
-<WRAP center round box important half column > ** Further useful links**+<WRAP center round box half column > ** Further useful links**
  
-  * [[https://docs.hpc.kaust.edu.sa/training/ibex_user_resources.html#ibex-user-resources|Ibex user resources]] 
   * [[https://docs.hpc.kaust.edu.sa/index.html| KSL Support Documentation ]]   * [[https://docs.hpc.kaust.edu.sa/index.html| KSL Support Documentation ]]
-  * [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/index.html|Ibex main index]] +  * **ibex@hpc.kaust.edu.sa** - email this address to create a service request 
-  * [[https://docs.hpc.kaust.edu.sa/soft_env/prog_env/modulesystem/ibex/datascience.html#data-science-on-ibex|Data Science on Ibex]]+  * https://kaust-ibex.slack.com - Use **#general** for simple queries 
 +</WRAP>
  
-/*   * [[https://kaust-supercomputing-lab.atlassian.net/l/cp/nM0wqF7a|Ibex Starting pack]]  */ 
  
  
-</WRAP> 
  
-<WRAP center round box important half column > {{:icon:twbs:link:w-25:h-auto:headset-vr.svg?nolink&}}** KVL Training on Data Science**+ 
 +<WRAP center round box half column > {{:icon:twbs:link:w-25:h-auto:headset-vr.svg?nolink&}}** KVL Training on Data Science** 
  
   * KVL often runs training on Data Science in collaboration with the KSL. Check out our current and past workshops for more: <wrap button>[[training:overview|Training Events]]</wrap>   * KVL often runs training on Data Science in collaboration with the KSL. Check out our current and past workshops for more: <wrap button>[[training:overview|Training Events]]</wrap>
Line 67: Line 107:
 </WRAP> </WRAP>
  
 +</WRAP>