{ "cells": [ { "cell_type": "markdown", "id": "badcf68f", "metadata": {}, "source": [ "# Human Connectome Project 7T fMRI Retinotopy Dataset\n", "\n", "This notebook shows how to automate the download and management of **Human Connectome Project (HCP) 7T fMRI dataset**. The data involved ultra-high 7 Tesla fMRI retinotopic mapping data in 181 healthy adults, with additional movie-watching, and resting-state runs (1.6-mm resolution and repetition time of 1s).\n", "\n", "\n", "The notebook covers five main stages: configuring the environment and AWS credentials for S3 access, loading validated subject IDs with complete retinotopy data, defining task parameters for retinotopy (RETBAR 1-2), movie-watching (MOVIE 1-4), and resting-state (REST 1-4) scans, checking the download status of existing data, and performing batch downloads from the [HCP OpenAccess S3 bucket](https://wiki.humanconnectome.org/docs/How%20to%20Get%20Access%20to%20the%20HCP%20OpenAccess%20Amazon%20S3%20Bucket.html). Each subject requires approximately 2-3 GB of storage across 10 scan runs. Note that HCP database access (free registration) is required, and AWS credentials should be configured before running the download scripts.\n", "\n", "\n", "Benson, N. C., Jamison, K. W., Arcaro, M. J., Vu, A. T., Glasser, M. F., Coalson, T. S., Van Essen, D. V., Yacoub, E., Ugurbil, K., Winawer, J., & Kay, K. (2018). **The Human Connectome Project 7 Tesla retinotopy dataset: Description and population receptive field analysis**. Journal of Vision, 18(13), 23. https://doi.org/10.1167/18.13.23\n" ] }, { "cell_type": "markdown", "id": "3e89b6a9", "metadata": {}, "source": [ "## Environment setup & dependencies\n", "\n", "Configure AWS credentials and import necessary libraries for CIFTI file handling and S3 data access." ] }, { "cell_type": "code", "execution_count": null, "id": "bb5002b7-3f84-4317-af20-b94c7da1ae26", "metadata": {}, "outputs": [], "source": [ "import os\n", "from nibabel import cifti2\n", "import numpy as np\n", "import itertools\n", "import nibabel as nib\n", "import argparse\n", "import boto3\n", "import sys\n", "import pickle\n", "\n", "\n", "aws_access_key_id='xxxx...'\n", "aws_secret_access_key='xxxx...'\n" ] }, { "cell_type": "markdown", "id": "4571d15b", "metadata": {}, "source": [ "## Subject ID configuration\n", "\n", "List of 184 validated HCP subject IDs with complete 7T retinotopy data." ] }, { "cell_type": "markdown", "id": "cf4d58aa", "metadata": {}, "source": [ "To get the 7T `subj_id.txt` :\n", "\n", "```shell\n", "100610\n", "102311\n", "102816\n", "104416\n", "105923\n", "108323\n", "109123\n", "111312\n", "111514\n", "114823\n", "115017\n", "115825\n", "116726\n", "118225\n", "125525\n", "126426\n", "128935\n", "130518\n", "131217\n", "131722\n", "134627\n", "134829\n", "135124\n", "137128\n", "140117\n", "144226\n", "145834\n", "146129\n", "146432\n", "146735\n", "146937\n", "148133\n", "150423\n", "155938\n", "156334\n", "157336\n", "158035\n", "158136\n", "159239\n", "162935\n", "164131\n", "164636\n", "165436\n", "167036\n", "167440\n", "169343\n", "169747\n", "171633\n", "172130\n", "173334\n", "175237\n", "176542\n", "177140\n", "177645\n", "177746\n", "178142\n", "178243\n", "178647\n", "180533\n", "181232\n", "181636\n", "182436\n", "182739\n", "185442\n", "186949\n", "187345\n", "191033\n", "191336\n", "191841\n", "192439\n", "192641\n", "193845\n", "195041\n", "196144\n", "197348\n", "198653\n", "199655\n", "200210\n", "200311\n", "200614\n", "201515\n", "203418\n", "204521\n", "205220\n", "209228\n", "212419\n", "214019\n", "214524\n", "221319\n", "233326\n", "239136\n", "246133\n", "249947\n", "251833\n", "257845\n", "263436\n", "283543\n", "318637\n", "320826\n", "330324\n", "346137\n", "352738\n", "360030\n", "365343\n", "380036\n", "381038\n", "389357\n", "393247\n", "395756\n", "397760\n", "406836\n", "412528\n", "429040\n", "436845\n", "463040\n", "467351\n", "473952\n", "525541\n", "536647\n", "541943\n", "547046\n", "550439\n", "552241\n", "562345\n", "572045\n", "573249\n", "581450\n", "585256\n", "601127\n", "617748\n", "627549\n", "638049\n", "654552\n", "671855\n", "680957\n", "690152\n", "706040\n", "724446\n", "725751\n", "732243\n", "745555\n", "751550\n", "757764\n", "765864\n", "770352\n", "771354\n", "782561\n", "783462\n", "789373\n", "814649\n", "818859\n", "825048\n", "826353\n", "833249\n", "859671\n", "861456\n", "871762\n", "872764\n", "878776\n", "878877\n", "898176\n", "899885\n", "901139\n", "905147\n", "910241\n", "926862\n", "927359\n", "942658\n", "951457\n", "958976\n", "966975\n", "995174\n", "```" ] }, { "cell_type": "markdown", "id": "a705d531", "metadata": {}, "source": [ "## Task & path definition\n", "\n", "Define task IDs for all scan types and configure local storage paths." ] }, { "cell_type": "code", "execution_count": 3, "id": "f8b937b3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Subjects : ['100610', '102311', '102816', '104416', '105923', '108323', '109123', '111312', '111514', '114823', '115017', '115825', '116726', '118225', '125525', '126426', '128935', '130518', '131217', '131722', '134627', '134829', '135124', '137128', '140117', '144226', '145834', '146129', '146432', '146735', '146937', '148133', '150423', '155938', '156334', '157336', '158035', '158136', '159239', '162935', '164131', '164636', '165436', '167036', '167440', '169343', '169747', '171633', '172130', '173334', '175237', '176542', '177140', '177645', '177746', '178142', '178243', '178647', '180533', '181232', '181636', '182436', '182739', '185442', '186949', '187345', '191033', '191336', '191841', '192439', '192641', '193845', '195041', '196144', '197348', '198653', '199655', '200210', '200311', '200614', '201515', '203418', '204521', '205220', '209228', '212419', '214019', '214524', '221319', '233326', '239136', '246133', '249947', '251833', '257845', '263436', '283543', '318637', '320826', '330324', '346137', '352738', '360030', '365343', '380036', '381038', '389357', '393247', '395756', '397760', '406836', '412528', '429040', '436845', '463040', '467351', '473952', '525541', '536647', '541943', '547046', '550439', '552241', '562345', '572045', '573249', '581450', '585256', '601127', '617748', '627549', '638049', '654552', '671855', '680957', '690152', '706040', '724446', '725751', '732243', '745555', '751550', '757764', '765864', '770352', '771354', '782561', '783462', '789373', '814649', '818859', '825048', '826353', '833249', '859671', '861456', '871762', '872764', '878776', '878877', '898176', '899885', '901139', '905147', '910241', '926862', '927359', '942658', '951457', '958976', '966975', '995174']\n", "Number of tasks : 10\n" ] } ], "source": [ "fileobj = open('subj_id.txt')\n", "data = fileobj.read()\n", "subject_ID = data.split('\\n')\n", "print('Subjects : ', subject_ID)\n", "\n", "task_ID_1 = ['tfMRI_RETBAR1_7T_AP',\n", " 'tfMRI_RETBAR2_7T_PA'\n", " ]\n", "\n", "task_ID_2 = ['tfMRI_MOVIE1_7T_AP',\n", " 'tfMRI_MOVIE2_7T_PA',\n", " 'tfMRI_MOVIE3_7T_PA',\n", " 'tfMRI_MOVIE4_7T_AP', \n", " ]\n", "\n", "task_ID_3 = ['rfMRI_REST1_7T_PA',\n", " 'rfMRI_REST2_7T_AP',\n", " 'rfMRI_REST3_7T_PA',\n", " 'rfMRI_REST4_7T_AP'\n", " ]\n", "\n", "task_ID = task_ID_1 + task_ID_2 + task_ID_3\n", "print('Number of tasks : ',len(task_ID))\n", "\n", "temp_root = '/scratch/nicogravel/HCP_data/temp/'\n", "file = '_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii'" ] }, { "cell_type": "markdown", "id": "269bd865", "metadata": {}, "source": [ "## Download status check\n", "\n", "Scan local directory to identify already-downloaded subjects and calculate storage requirements." ] }, { "cell_type": "code", "execution_count": 4, "id": "4dd5f86d", "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "subject: 100610 , size(MB): 5123.201560974121\n", "subject: 102311 , size(MB): 5123.201560974121\n", "subject: 102816 , size(MB): 5123.201560974121\n", "subject: 104416 , size(MB): 5123.201560974121\n", "subject: 105923 , size(MB): 5123.201560974121\n", "subject: 108323 , size(MB): 5123.201560974121\n", "subject: 109123 , size(MB): 5123.201560974121\n", "subject: 111312 , size(MB): 4536.369209289551\n", "subject: 111514 , size(MB): 5120.202827453613\n", "subject: 114823 , size(MB): 5120.202827453613\n", "subject: 115017 , size(MB): 5120.202827453613\n", "subject: 115825 , size(MB): 5120.202827453613\n", "subject: 116726 , size(MB): 5120.202827453613\n", "subject: 118225 , size(MB): 5120.202827453613\n", "subject: 125525 , size(MB): 5120.202827453613\n", "subject: 126426 , size(MB): 5120.202827453613\n", "subject: 128935 , size(MB): 5120.202827453613\n", "subject: 130518 , size(MB): 5120.202827453613\n", "subject: 131217 , size(MB): 5120.202827453613\n", "subject: 131722 , size(MB): 5120.202827453613\n", "subject: 134627 , size(MB): 5120.202827453613\n", "subject: 134829 , size(MB): 5120.202827453613\n", "subject: 135124 , size(MB): 5120.202827453613\n", "subject: 137128 , size(MB): 5120.202827453613\n", "subject: 140117 , size(MB): 5120.202827453613\n", "subject: 144226 , size(MB): 5120.202827453613\n", "subject: 145834 , size(MB): 5120.202827453613\n", "subject: 146129 , size(MB): 5120.202827453613\n", "subject: 146432 , size(MB): 5120.202827453613\n", "subject: 146735 , size(MB): 5120.202827453613\n", "subject: 146937 , size(MB): 5120.202827453613\n", "subject: 148133 , size(MB): 5120.202827453613\n", "subject: 150423 , size(MB): 5120.202827453613\n", "subject: 155938 , size(MB): 5120.202827453613\n", "subject: 156334 , size(MB): 5120.202827453613\n", "subject: 157336 , size(MB): 5120.202827453613\n", "subject: 158035 , size(MB): 5120.202827453613\n", "subject: 158136 , size(MB): 5120.202827453613\n", "subject: 159239 , size(MB): 5120.202827453613\n", "subject: 162935 , size(MB): 5120.202827453613\n", "subject: 164131 , size(MB): 5120.202827453613\n", "subject: 164636 , size(MB): 5120.202827453613\n", "subject: 165436 , size(MB): 5120.202827453613\n", "subject: 167036 , size(MB): 5120.202827453613\n", "subject: 167440 , size(MB): 5120.202827453613\n", "subject: 169343 , size(MB): 5120.202827453613\n", "subject: 169747 , size(MB): 5120.202827453613\n", "subject: 171633 , size(MB): 5120.202827453613\n", "subject: 172130 , size(MB): 5120.202827453613\n", "subject: 173334 , size(MB): 5120.202827453613\n", "subject: 175237 , size(MB): 5120.202827453613\n", "subject: 176542 , size(MB): 5120.202827453613\n", "subject: 177140 , size(MB): 5120.202827453613\n", "subject: 177645 , size(MB): 5120.202827453613\n", "subject: 177746 , size(MB): 5120.202827453613\n", "subject: 178142 , size(MB): 5120.202827453613\n", "subject: 178243 , size(MB): 5120.202827453613\n", "subject: 178647 , size(MB): 5120.202827453613\n", "subject: 180533 , size(MB): 5120.202827453613\n", "subject: 181232 , size(MB): 5120.202827453613\n", "subject: 181636 , size(MB): 1590.863899230957\n", "subject: 182436 , size(MB): 5120.202827453613\n", "subject: 182739 , size(MB): 5120.202827453613\n", "subject: 185442 , size(MB): 5120.202827453613\n", "subject: 186949 , size(MB): 5120.202827453613\n", "subject: 187345 , size(MB): 5120.202827453613\n", "subject: 191033 , size(MB): 5120.202827453613\n", "subject: 191336 , size(MB): 5120.202827453613\n", "subject: 191841 , size(MB): 5120.202827453613\n", "subject: 192439 , size(MB): 5120.202827453613\n", "subject: 192641 , size(MB): 5120.202827453613\n", "subject: 193845 , size(MB): 5120.202827453613\n", "subject: 195041 , size(MB): 5120.202827453613\n", "subject: 196144 , size(MB): 5120.202827453613\n", "subject: 197348 , size(MB): 5120.202827453613\n", "subject: 198653 , size(MB): 5120.202827453613\n", "subject: 199655 , size(MB): 5120.202827453613\n", "subject: 200210 , size(MB): 5120.202827453613\n", "subject: 200311 , size(MB): 5120.202827453613\n", "subject: 200614 , size(MB): 5120.202827453613\n", "subject: 201515 , size(MB): 5120.202827453613\n", "subject: 203418 , size(MB): 5120.202827453613\n", "subject: 204521 , size(MB): 5120.202827453613\n", "subject: 205220 , size(MB): 5120.202827453613\n", "subject: 209228 , size(MB): 5120.202827453613\n", "subject: 212419 , size(MB): 5120.202827453613\n", "subject: 214019 , size(MB): 5120.202827453613\n", "subject: 214524 , size(MB): 5120.202827453613\n", "subject: 221319 , size(MB): 5120.202827453613\n", "subject: 233326 , size(MB): 5120.202827453613\n", "subject: 239136 , size(MB): 5120.202827453613\n", "subject: 246133 , size(MB): 5120.202827453613\n", "subject: 249947 , size(MB): 5120.202827453613\n", "subject: 251833 , size(MB): 5120.202827453613\n", "subject: 257845 , size(MB): 5120.202827453613\n", "subject: 263436 , size(MB): 5120.202827453613\n", "subject: 283543 , size(MB): 5120.202827453613\n", "subject: 318637 , size(MB): 5120.202827453613\n", "subject: 320826 , size(MB): 5120.202827453613\n", "subject: 330324 , size(MB): 5120.202827453613\n", "subject: 346137 , size(MB): 5120.202827453613\n", "subject: 352738 , size(MB): 5120.202827453613\n", "subject: 360030 , size(MB): 5120.202827453613\n", "subject: 365343 , size(MB): 5120.202827453613\n", "subject: 380036 , size(MB): 5120.202827453613\n", "subject: 381038 , size(MB): 5120.202827453613\n", "subject: 389357 , size(MB): 5120.202827453613\n", "subject: 393247 , size(MB): 5120.202827453613\n", "subject: 395756 , size(MB): 5120.202827453613\n", "subject: 397760 , size(MB): 5120.202827453613\n", "subject: 406836 , size(MB): 5120.202827453613\n", "subject: 412528 , size(MB): 5120.202827453613\n", "subject: 429040 , size(MB): 5120.202827453613\n", "subject: 436845 , size(MB): 5120.202827453613\n", "subject: 463040 , size(MB): 5120.202827453613\n", "subject: 467351 , size(MB): 5120.202827453613\n", "subject: 473952 , size(MB): 0.0\n", "subject: 525541 , size(MB): 5120.202827453613\n", "subject: 536647 , size(MB): 1590.863899230957\n", "subject: 541943 , size(MB): 5120.202827453613\n", "subject: 547046 , size(MB): 5120.202827453613\n", "subject: 550439 , size(MB): 5120.202827453613\n", "subject: 552241 , size(MB): 1590.863899230957\n", "subject: 562345 , size(MB): 5120.202827453613\n", "subject: 572045 , size(MB): 5120.202827453613\n", "subject: 573249 , size(MB): 5120.202827453613\n", "subject: 581450 , size(MB): 5120.202827453613\n", "subject: 585256 , size(MB): 992.6682052612305\n", "subject: 601127 , size(MB): 5120.202827453613\n", "subject: 617748 , size(MB): 5120.202827453613\n", "subject: 627549 , size(MB): 5120.202827453613\n", "subject: 638049 , size(MB): 5120.202827453613\n", "subject: 654552 , size(MB): 5120.202827453613\n", "subject: 671855 , size(MB): 5120.202827453613\n", "subject: 680957 , size(MB): 5120.202827453613\n", "subject: 690152 , size(MB): 5120.202827453613\n", "subject: 706040 , size(MB): 5120.202827453613\n", "subject: 724446 , size(MB): 5120.202827453613\n", "subject: 725751 , size(MB): 5120.202827453613\n", "subject: 732243 , size(MB): 5120.202827453613\n", "subject: 745555 , size(MB): 0.0\n", "subject: 878877 , size(MB): 5120.202827453613\n", "subject: 898176 , size(MB): 5120.202827453613\n", "subject: 899885 , size(MB): 5120.202827453613\n", "subject: 901139 , size(MB): 5120.202827453613\n", "subject: 905147 , size(MB): 5120.202827453613\n", "subject: 910241 , size(MB): 5120.202827453613\n", "subject: 926862 , size(MB): 5120.202827453613\n", "subject: 927359 , size(MB): 5120.202827453613\n", "subject: 942658 , size(MB): 5120.202827453613\n", "subject: 951457 , size(MB): 5111.747848510742\n", "subject: 958976 , size(MB): 5120.202827453613\n", "subject: 966975 , size(MB): 5120.202827453613\n", "subject: 995174 , size(MB): 5120.202827453613\n", "Subject downloaded : 154 , Subject to download : 18\n" ] } ], "source": [ "n_subj = 0\n", "x_subj = 0\n", "for subj in range(len(subject_ID)):\n", " isdir = os.path.isdir(temp_root + subject_ID[subj] + '/') \n", " #print(isdir) \n", " if isdir == True: \n", " folder = temp_root + subject_ID[subj] \n", " # get size\n", " size = 0\n", " for ele in os.scandir(folder):\n", " size+=os.path.getsize(ele)\n", " print('subject: ', subject_ID[subj], ', size(MB): ', size/1024**2) \n", " n_subj = + n_subj + 1\n", " if isdir == False: \n", " x_subj = + x_subj + 1\n", "print('Subject downloaded : ',n_subj, ', Subject to download : ', x_subj)\n" ] }, { "cell_type": "markdown", "id": "9d7957eb", "metadata": {}, "source": [ "## Batch download\n", "\n", "Download CIFTI timeseries files from HCP OpenAccess bucket for all missing subjects and task runs." ] }, { "cell_type": "code", "execution_count": 4, "id": "485da456", "metadata": { "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "subject: 100610 , size(MB): 5123.201560974121\n", "subject: 102311 , size(MB): 5123.201560974121\n", "subject: 102816 , size(MB): 5123.201560974121\n", "subject: 104416 , size(MB): 5123.201560974121\n", "subject: 105923 , size(MB): 5123.201560974121\n", "subject: 108323 , size(MB): 5123.201560974121\n", "subject: 109123 , size(MB): 5123.201560974121\n", "subject: 111312 , size(MB): 4536.369209289551\n", "subject: 111514 , size(MB): 5120.202827453613\n", "subject: 114823 , size(MB): 5120.202827453613\n", "subject: 115017 , size(MB): 5120.202827453613\n", "subject: 115825 , size(MB): 5120.202827453613\n", "subject: 116726 , size(MB): 5120.202827453613\n", "subject: 118225 , size(MB): 5120.202827453613\n", "subject: 125525 , size(MB): 5120.202827453613\n", "subject: 126426 , size(MB): 5120.202827453613\n", "subject: 128935 , size(MB): 5120.202827453613\n", "subject: 130518 , size(MB): 5120.202827453613\n", "subject: 131217 , size(MB): 5120.202827453613\n", "subject: 131722 , size(MB): 5120.202827453613\n", "subject: 134627 , size(MB): 5120.202827453613\n", "subject: 134829 , size(MB): 5120.202827453613\n", "subject: 135124 , size(MB): 5120.202827453613\n", "subject: 137128 , size(MB): 5120.202827453613\n", "subject: 140117 , size(MB): 5120.202827453613\n", "subject: 144226 , size(MB): 5120.202827453613\n", "subject: 145834 , size(MB): 5120.202827453613\n", "subject: 146129 , size(MB): 5120.202827453613\n", "subject: 146432 , size(MB): 5120.202827453613\n", "subject: 146735 , size(MB): 5120.202827453613\n", "subject: 146937 , size(MB): 5120.202827453613\n", "subject: 148133 , size(MB): 5120.202827453613\n", "subject: 150423 , size(MB): 5120.202827453613\n", "subject: 155938 , size(MB): 5120.202827453613\n", "subject: 156334 , size(MB): 5120.202827453613\n", "subject: 157336 , size(MB): 5120.202827453613\n", "subject: 158035 , size(MB): 5120.202827453613\n", "subject: 158136 , size(MB): 5120.202827453613\n", "subject: 159239 , size(MB): 5120.202827453613\n", "subject: 162935 , size(MB): 5120.202827453613\n", "subject: 164131 , size(MB): 5120.202827453613\n", "subject: 164636 , size(MB): 5120.202827453613\n", "subject: 165436 , size(MB): 5120.202827453613\n", "subject: 167036 , size(MB): 5120.202827453613\n", "subject: 167440 , size(MB): 5120.202827453613\n", "subject: 169343 , size(MB): 5120.202827453613\n", "subject: 169747 , size(MB): 5120.202827453613\n", "subject: 171633 , size(MB): 5120.202827453613\n", "subject: 172130 , size(MB): 5120.202827453613\n", "subject: 173334 , size(MB): 5120.202827453613\n", "subject: 175237 , size(MB): 5120.202827453613\n", "subject: 176542 , size(MB): 5120.202827453613\n", "subject: 177140 , size(MB): 5120.202827453613\n", "subject: 177645 , size(MB): 5120.202827453613\n", "subject: 177746 , size(MB): 5120.202827453613\n", "subject: 178142 , size(MB): 5120.202827453613\n", "subject: 178243 , size(MB): 5120.202827453613\n", "subject: 178647 , size(MB): 5120.202827453613\n", "subject: 180533 , size(MB): 5120.202827453613\n", "subject: 181232 , size(MB): 5120.202827453613\n", "subject: 181636 , size(MB): 1590.863899230957\n", "subject: 182436 , size(MB): 5120.202827453613\n", "subject: 182739 , size(MB): 5120.202827453613\n", "subject: 185442 , size(MB): 5120.202827453613\n", "subject: 186949 , size(MB): 5120.202827453613\n", "subject: 187345 , size(MB): 5120.202827453613\n", "subject: 191033 , size(MB): 5120.202827453613\n", "subject: 191336 , size(MB): 5120.202827453613\n", "subject: 191841 , size(MB): 5120.202827453613\n", "subject: 192439 , size(MB): 5120.202827453613\n", "subject: 192641 , size(MB): 5120.202827453613\n", "subject: 193845 , size(MB): 5120.202827453613\n", "subject: 195041 , size(MB): 5120.202827453613\n", "subject: 196144 , size(MB): 5120.202827453613\n", "subject: 197348 , size(MB): 5120.202827453613\n", "subject: 198653 , size(MB): 5120.202827453613\n", "subject: 199655 , size(MB): 5120.202827453613\n", "subject: 200210 , size(MB): 5120.202827453613\n", "subject: 200311 , size(MB): 5120.202827453613\n", "subject: 200614 , size(MB): 5120.202827453613\n", "subject: 201515 , size(MB): 5120.202827453613\n", "subject: 203418 , size(MB): 5120.202827453613\n", "subject: 204521 , size(MB): 5120.202827453613\n", "subject: 205220 , size(MB): 5120.202827453613\n", "subject: 209228 , size(MB): 5120.202827453613\n", "subject: 212419 , size(MB): 5120.202827453613\n", "subject: 214019 , size(MB): 5120.202827453613\n", "subject: 214524 , size(MB): 5120.202827453613\n", "subject: 221319 , size(MB): 5120.202827453613\n", "subject: 233326 , size(MB): 5120.202827453613\n", "subject: 239136 , size(MB): 5120.202827453613\n", "subject: 246133 , size(MB): 5120.202827453613\n", "subject: 249947 , size(MB): 5120.202827453613\n", "subject: 251833 , size(MB): 5120.202827453613\n", "subject: 257845 , size(MB): 5120.202827453613\n", "subject: 263436 , size(MB): 5120.202827453613\n", "subject: 283543 , size(MB): 5120.202827453613\n", "subject: 318637 , size(MB): 5120.202827453613\n", "subject: 320826 , size(MB): 5120.202827453613\n", "subject: 330324 , size(MB): 5120.202827453613\n", "subject: 346137 , size(MB): 5120.202827453613\n", "subject: 352738 , size(MB): 5120.202827453613\n", "subject: 360030 , size(MB): 5120.202827453613\n", "subject: 365343 , size(MB): 5120.202827453613\n", "subject: 380036 , size(MB): 5120.202827453613\n", "subject: 381038 , size(MB): 5120.202827453613\n", "subject: 389357 , size(MB): 5120.202827453613\n", "subject: 393247 , size(MB): 5120.202827453613\n", "subject: 395756 , size(MB): 5120.202827453613\n", "subject: 397760 , size(MB): 5120.202827453613\n", "subject: 406836 , size(MB): 5120.202827453613\n", "subject: 412528 , size(MB): 5120.202827453613\n", "subject: 429040 , size(MB): 5120.202827453613\n", "subject: 436845 , size(MB): 5120.202827453613\n", "subject: 463040 , size(MB): 5120.202827453613\n", "subject: 467351 , size(MB): 5120.202827453613\n", "subject: 473952 , size(MB): 0.0\n", "subject: 525541 , size(MB): 5120.202827453613\n", "subject: 536647 , size(MB): 1590.863899230957\n", "subject: 541943 , size(MB): 5120.202827453613\n", "subject: 547046 , size(MB): 5120.202827453613\n", "subject: 550439 , size(MB): 5120.202827453613\n", "subject: 552241 , size(MB): 1590.863899230957\n", "subject: 562345 , size(MB): 5120.202827453613\n", "subject: 572045 , size(MB): 5120.202827453613\n", "subject: 573249 , size(MB): 5120.202827453613\n", "subject: 581450 , size(MB): 5120.202827453613\n", "subject: 585256 , size(MB): 992.6682052612305\n", "subject: 601127 , size(MB): 5120.202827453613\n", "subject: 617748 , size(MB): 5120.202827453613\n", "subject: 627549 , size(MB): 5120.202827453613\n", "subject: 638049 , size(MB): 5120.202827453613\n", "subject: 654552 , size(MB): 5120.202827453613\n", "subject: 671855 , size(MB): 5120.202827453613\n", "subject: 680957 , size(MB): 5120.202827453613\n", "subject: 690152 , size(MB): 5120.202827453613\n", "subject: 706040 , size(MB): 5120.202827453613\n", "subject: 724446 , size(MB): 5120.202827453613\n", "subject: 725751 , size(MB): 5120.202827453613\n", "subject: 732243 , size(MB): 5120.202827453613\n", "subject: 745555 , size(MB): 0.0\n", "subject: 751550 , size(MB): 5120.202827453613\n", "subject: 757764 , size(MB): 5120.202827453613\n", "subject: 765864 , size(MB): 5120.202827453613\n", "subject: 770352 , size(MB): 5120.202827453613\n", "subject: 771354 , size(MB): 5120.202827453613\n", "subject: 782561 , size(MB): 5120.202827453613\n", "subject: 783462 , size(MB): 5120.202827453613\n", "subject: 789373 , size(MB): 5120.202827453613\n", "subject: 814649 , size(MB): 5120.202827453613\n", "subject: 818859 , size(MB): 5120.202827453613\n", "subject: 825048 , size(MB): 392.5213623046875\n", "Processing data for subject : 826353 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Processing data for subject : 826353 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/826353/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : rfMRI_REST1_7T_PA\n", "HCP_1200/826353/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : rfMRI_REST2_7T_AP\n", "HCP_1200/826353/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : rfMRI_REST3_7T_PA\n", "HCP_1200/826353/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 826353 task : rfMRI_REST4_7T_AP\n", "HCP_1200/826353/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/826353/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/833249/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : rfMRI_REST1_7T_PA\n", "HCP_1200/833249/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : rfMRI_REST2_7T_AP\n", "HCP_1200/833249/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : rfMRI_REST3_7T_PA\n", "HCP_1200/833249/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 833249 task : rfMRI_REST4_7T_AP\n", "HCP_1200/833249/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/833249/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/859671/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : rfMRI_REST1_7T_PA\n", "HCP_1200/859671/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : rfMRI_REST2_7T_AP\n", "HCP_1200/859671/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : rfMRI_REST3_7T_PA\n", "HCP_1200/859671/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 859671 task : rfMRI_REST4_7T_AP\n", "HCP_1200/859671/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/859671/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/861456/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_RETBAR2_7T_PA\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "HCP_1200/861456/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/861456/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/861456/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/861456/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/861456/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : rfMRI_REST1_7T_PA\n", "HCP_1200/861456/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : rfMRI_REST2_7T_AP\n", "HCP_1200/861456/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : rfMRI_REST3_7T_PA\n", "HCP_1200/861456/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 861456 task : rfMRI_REST4_7T_AP\n", "HCP_1200/861456/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/861456/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/871762/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : rfMRI_REST1_7T_PA\n", "HCP_1200/871762/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : rfMRI_REST2_7T_AP\n", "HCP_1200/871762/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : rfMRI_REST3_7T_PA\n", "HCP_1200/871762/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 871762 task : rfMRI_REST4_7T_AP\n", "HCP_1200/871762/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/871762/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/872764/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : rfMRI_REST1_7T_PA\n", "HCP_1200/872764/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : rfMRI_REST2_7T_AP\n", "HCP_1200/872764/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 872764 task : rfMRI_REST3_7T_PA\n", "HCP_1200/872764/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Processing data for subject : 872764 task : rfMRI_REST4_7T_AP\n", "HCP_1200/872764/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/872764/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_RETBAR1_7T_AP\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_RETBAR1_7T_AP/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_RETBAR1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_RETBAR2_7T_PA\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_RETBAR2_7T_PA/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_RETBAR2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_MOVIE1_7T_AP\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_MOVIE1_7T_AP/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_MOVIE1_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_MOVIE2_7T_PA\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_MOVIE2_7T_PA/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_MOVIE2_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_MOVIE3_7T_PA\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_MOVIE3_7T_PA/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_MOVIE3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : tfMRI_MOVIE4_7T_AP\n", "HCP_1200/878776/MNINonLinear/Results/tfMRI_MOVIE4_7T_AP/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/tfMRI_MOVIE4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : rfMRI_REST1_7T_PA\n", "HCP_1200/878776/MNINonLinear/Results/rfMRI_REST1_7T_PA/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/rfMRI_REST1_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : rfMRI_REST2_7T_AP\n", "HCP_1200/878776/MNINonLinear/Results/rfMRI_REST2_7T_AP/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/rfMRI_REST2_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : rfMRI_REST3_7T_PA\n", "HCP_1200/878776/MNINonLinear/Results/rfMRI_REST3_7T_PA/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/rfMRI_REST3_7T_PA_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "Processing data for subject : 878776 task : rfMRI_REST4_7T_AP\n", "HCP_1200/878776/MNINonLinear/Results/rfMRI_REST4_7T_AP/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "/scratch/nicogravel/HCP_data/temp/878776/rfMRI_REST4_7T_AP_Atlas_1.6mm_MSMAll_hp2000_clean.dtseries.nii\n", "Downloading...\n", "subject: 878877 , size(MB): 5120.202827453613\n", "subject: 898176 , size(MB): 5120.202827453613\n", "subject: 899885 , size(MB): 5120.202827453613\n", "subject: 901139 , size(MB): 5120.202827453613\n", "subject: 905147 , size(MB): 5120.202827453613\n", "subject: 910241 , size(MB): 5120.202827453613\n", "subject: 926862 , size(MB): 5120.202827453613\n", "subject: 927359 , size(MB): 5120.202827453613\n", "subject: 942658 , size(MB): 5120.202827453613\n", "subject: 951457 , size(MB): 5111.747848510742\n", "subject: 958976 , size(MB): 5120.202827453613\n", "subject: 966975 , size(MB): 5120.202827453613\n", "subject: 995174 , size(MB): 5120.202827453613\n" ] } ], "source": [ "n_subj = 0\n", "x_subj = 0\n", "\n", "for subj in range(len(subject_ID)):\n", " isdir = os.path.isdir(temp_root + subject_ID[subj] + '/') \n", " \n", " \n", " if isdir == True: \n", " folder = temp_root + subject_ID[subj] \n", " # get size\n", " size = 0\n", " for ele in os.scandir(folder):\n", " size+=os.path.getsize(ele)\n", " print('subject: ', subject_ID[subj], ', size(MB): ', size/1024**2) \n", " n_subj = + n_subj + 1\n", " \n", " if isdir == False: \n", " os.mkdir(temp_root + subject_ID[subj])\n", " for run in range(len(task_ID)):\n", " print('Processing data for subject :', subject_ID[subj], 'task :', task_ID[run])\n", " dbhcp = 'HCP_1200/' + subject_ID[subj] + '/MNINonLinear/Results/'\n", " s3_client = boto3.client('s3',\n", " aws_access_key_id = aws_access_key_id,\n", " aws_secret_access_key = aws_secret_access_key)\n", " s3_bucket_name = 'hcp-openaccess'\n", " s3_path = dbhcp + task_ID[run] + '/' + task_ID[run] + file\n", " print(s3_path)\n", " temp_save_path = temp_root + subject_ID[subj] + '/' + task_ID[run] + file\n", " print(temp_save_path)\n", " print('Downloading...')\n", " s3_client.download_file(Bucket=s3_bucket_name, Key=s3_path, Filename=temp_save_path)\n", " sub_file = temp_save_path" ] } ], "metadata": { "kernelspec": { "display_name": "lamidec", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.19" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "position": { "height": "144px", "left": "710px", "right": "20px", "top": "121px", "width": "350px" }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 5 }