feat: add multi core job example

Closes #4

2_multi_core_job/README.md

+# Multi core jobs
+A multi core job is a job that splits the computation to multiple cores. This type of job is the most suitable and most common ones to run on Lundgren. This includes optimization problems and heavy computations.
+
+## How to run
+To run the example do the following steps:
+1. Log in to Lundgren
+2. Change directory to the example code
+3. Run `sbatch multi_core_job.sh`
+4. Check queue status by running `squeue`
+5. When the job is completed check the file _multi_core_job.log_
+
+Try changing the number of cpus in _multi_core_job.sh_ and see the changes in processing time.
+
+## Detailed description of the example
+The batch script is the main file for the job allocation and preparation. Inside the python script a few environmental variables are fetched and printed out.
+
+### The batch script
+The batch script, multi_core_job.sh_, contains three sections. The first section contains input arguments to the Slurm scheduler. The second section loads Python into environment so it is accessible and lastly the a job step is performed.
+
+The input arguments are defined with a comment beginning with SBATCH followed by the argument key and value. For easier readablility the -- method is used.
+
+- __job-name:__ The name of the job is set to demo_multi_core
+- __time:__ The requeted time is set to 5 minutes, _00:05:00_
+- __ntasks:__ The number of tasks to be performed in this job is set to _1_.
+- __cpus-per-task:__ The requested number of cores per task is set to _2_
+- __mem:__ The requested memory is set to _50 MB_
+- __output:__ The standard output should be sent to the file multi_core_job.log_
+
+Python needs to be loaded into the environment in order to be accessible this is done in the next step with the __module__ command.
+
+The job step with the single task is allocated and performed with the __srun__ command.
+
+#### The python script
+The python script represents the taskt to be done. In this case the task is to wait a random time and print the waiting is done.
+
+The environment variable __SLURM_CPUS_PER_TASK__ is used to restrict the worker pool to the allocated number of cores.
+
+### How to set the number of cores in different programing languages and softwares
+Most programming languages and softwares tries to make use of all cores that are available. This can lead to an oversubscription on the resources. On a shared resource one must match the maximum used resources with the allocated ones. This section gives a reference in how to do it in commonly used softwares.
+
+- __CPLEX:__ Use the parameter _global thread count_. Read more in the [documentation](https://www.ibm.com/docs/en/icos/22.1.2?topic=parameters-global-thread-count)
+- __Gurobi:__ Use the configuration parameter _ThreadLimit_. Read more in the [documentation](https://docs.gurobi.com/projects/optimizer/en/current/reference/parameters.html#threadlimit)
+- __MATLAB:__ Create a instance of the parpool object with the _poolsize_ set to the number of cores and use the pool when running in parallell. Read more in the [documentation](https://se.mathworks.com/help/parallel-computing/parpool.html)
+- __Python:__  If the multiprocessing package is used, create an instance of the pool class with the _processes_ set to the number of cores and use the pool when running in parallell. Read more in the [documentation](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool)

2_multi_core_job/multi_core_job.sh

+#! /bin/bash
+#SBATCH --job-name=demo_multi_core
+#SBATCH --time=00:05:00
+#SBATCH --ntasks=1
+#SBATCH --cpus-per-task=2
+#SBATCH --mem-per-cpu=50MB
+#SBATCH --output=multi_core_job.log
+
+# Loading Python into the environment
+module load python/anaconda3-2024.02-3.11.7
+
+# Start job stage
+srun python multi_core_task.py
\ No newline at end of file

2_multi_core_job/multi_core_task.py

+from datetime import datetime
+from multiprocessing import Pool
+import logging
+import os
+import random
+import time
+
+logger = logging.getLogger(__name__)
+
+def sleep(input):
+    time.sleep(input[1])
+    logger.info('Task %d done.',input[0])
+
+def main():
+    # Read environment variables.
+    NUMBER_OF_CORES = os.environ.get('SLURM_CPUS_PER_TASK','Unknown')
+    if NUMBER_OF_CORES in 'Unknown':
+        logger.error('Unkown number of cores, exiting.')
+        return
+
+    NUMBER_OF_CORES = int(NUMBER_OF_CORES)
+    logger.info('Running program with %d cores.',NUMBER_OF_CORES)
+
+    # Creating a list of tasks to be performed
+    # This represents the calculations
+    random.seed(1)
+    tasks = []
+    total_time = 0
+    for i in range(10):
+        time = random.randrange(1,29)
+        tasks.append((i, time))
+        total_time = total_time + time
+
+    # Creating a multiprocessing pool to perform the tasks
+    pool = Pool(processes=NUMBER_OF_CORES)
+
+    # Running submitting the tasks to the worker pool
+    tic = datetime.now()
+    logger.info('Submitting tasks to pool.')
+    pool.map(sleep, tasks)
+    toc = datetime.now()
+
+    logger.info('All tasks are done, took %d seconds, compared to %d seconds with single thread.',
+        (toc-tic).seconds, total_time)
+    
+
+if __name__ == '__main__':
+    logging.basicConfig(level=logging.INFO)
+    main()

README.md

@@ -21,3 +21,8 @@ A single core job is a job with only a single thread. This type of job is used w
 A simple example could be a data parser that reads a file and transforms it into a more suitable format.
 
 Learn more about the [example](https://gitlab.liu.se/rasri17/lundgren-examples/-/tree/main/1_single_core_job).
+
+#### Example 2 - Mutli core job
+A multi core job is a job that splits the computation to multiple cores. This type of job is the most suitable and most common ones to run on Lundgren. This includes optimization problems and heavy computations.
+
+Learn more about the [example](https://gitlab.liu.se/rasri17/lundgren-examples/-/tree/main/2_multi_core_job).
\ No newline at end of file