Group's Computer Architecture
Each group member (user) is attributed a computer (that we will refer to as the "home computer" in the following) where all there files will be located. Only the files of the user on its home computer will be automatically backed up. The output files of any calculation ran on a computer other than the home computer will need to be moved back to the home computer to be backed up. Here is the list of the computers with their user and specifications:
Computer | User | #CPU | #GPU | Memory | Disk |
---|---|---|---|---|---|
tp-linux1 | 64 | 2 | 254Go | 4x10To | |
tp-linux2 | Julien | 64 | 2 | 254Go | 4x10To |
tp-linux3 | 64 | 2 | 254Go | 4x10To | |
tp-linux4 | 64 | 2 | 254Go | 4x10To | |
tp-linux5 | Tom Penfold | 6 | 0 | 64Go | 4.5To |
tp-linux6 | 16 | 0 | 32Go | 3.0To/5.0To | |
tp-linux7 | 36 | 0 | 64Go | 470Go/854Go | |
tp-linux8 | 64 | 0 | 64Go | 916Go/917Go/917Go/917Go | |
tp-linux9 | Sneha | 64 | 0 | 64Go | 916Go/917Go/1.8To/1.8To |
tp-linux10 | 64 | 0 | 64Go | 916Go/917Go/1.8To/1.8To | |
tp-linux11 | 16 | 0 | 64Go | 3.6To/2.0To | |
tp-linux12 | Miriam | 48 | 0 | 128Go | 3.6To/2.0To |
tp-linux13 | Tudur | 64 | 0 | 64Go | 911Go/3x20To |
tp-linux14 | Julia | 64 | 0 | 64Go | 916Go/19To/3.6To/3.6To |
tp-linux15 | Shawana | 64 | 2 | 128Go | 1.8To/1.8To/457Go |
tp-linux16 | 64 | 2 | 128Go | 1.8To/1.8To/457Go | |
tp-linux17 | Clelia | 64 | 2 | 128Go | 1.8To/1.8To/457Go |
tp-linux18 | 64 | 2 | 128Go | 1.8To/1.8To/457Go | |
tp-linux19 | Tom Pope | 64 | 2 | 128Go | 1.8To/1.8To/457Go | tp-linux20 | 64 | 2 | 128Go | 1.8To/1.8To/457Go |
Backing up your data
Ensuring your data is backed up is your responsibility. This should be checked regularly to ensure there are no problems if disk failures arise.
Backup Schedule
Day of the Week | Backup1 | Backup2 |
---|---|---|
Saturday | Tom Penfold | Tom Pope |
Sunday | Julien | Sneha |
Monday | Clelia | (empty) [Luke] |
Tuesday | Julia | Shawana |
Wednesday | Miriam | Adam |
Thursday | (empty) [Beth] | Tudur |
Backup Information
- Automatic backup occurs every as per the schedule above.
- All your data on your home directory (~/ or /home/${USER}) of your "home computer" is backed up. To make it more efficient, the group is split into two, and users will be backed up either on /backup1 or /backup2.
- Deleted files from your machines will be removed from /backup?/${USER}/ during the next backing up. This means that this is not a way of archiving your old project.
- Temporary files and system files will NOT be backed up. (See /software/group-utilities/backup-exceptions.list for the full list of files that will not be backed up).
- This automatic backup gives you a whole fortnight to get your files back in case something happens. Please note that the back-up will be done using BorgBackup in the form of a hashed archive.
Check Backup Status
- In order to check the status of your automatic backup -- and please do so every now and then! -- you can type from your "home computer":
borg list /mnt/backupX/#USER
- Where backupX is the folder your data are saved into (see table above) and #USER is your username.
List Files in Backup
- To list all the files/folders within a save, simply use the same command but specifying the save in addition:
borg list /mnt/backup1/tom::tp-linux5-2022-11-07-tom
- Borg list can also take a specific path as the second argument if you want to check whether this path is present in the archive.
- The leading backslash doesn't seem to be present within the archive – so try home/myname/... rather than /home/myname/...
Recover Data
- If you need to recover some data, the command is extract -- feel free to ask postdocs/Tom if you are not sure:
borg extract /mnt/backup1/tom::tp-linux5-2022-11-07-tom home/stolenfigures
- The above command would extract the folder "stolenfigures" to the current folder. For more info, please give the BorgBackup manual a look!
- (You can use the option --dry-run to check whether the command will work before actually performing an extraction.)
Directories and Locations
- Software: /software
- Templates: /software/group-utilities/Templates
- Scripts: A lot of homemade scripts on /software/group-utilities/Scripts . Scripts are automatically sourced in the common group .bashrc file (see section Common .bashrc file) and are available to be used simply by typing their name. A complete description of the scripts can be found in the README file provided in the directory. You are encouraged to participate to this directory by sending scripts you are using to one of the PostDoc (for writing permission reasons) to make them available for everyone! Please also provided a README entry for your script!
- You can also find a variety of useful stuff such as Templates for presentations slides, posters and Ph.D. thesis on /software/group-utilities/Templates
- Backup1: /backup1
- Backup2: /backup2
- Archive: /archive
- Former Group Member Data: /software/data_former_group_member (on TP-NAS)
Additional Information
- Each user is given a OneDrive storage space of 5To by the University.
- Most machines are within the office cabled network (purple cables) and plugged into a Gigabit network switch.
- Only tp-linux5 and tp-linux3 are located in Tom Penfold's office.