Merge remote-tracking branch 'cpp-lln-lab/master'

Author: CerenB

.gitignore

@@ -6,16 +6,17 @@
 *octave-workspace
 
 # exclude content of logfiles folders
-tests/output/*
-manualTests/output/*
-output/*
+*output*
 *.tsv
 *.mat
 
 # exclude temp files from tests and coverage
-tests/coverage*
-tests/test_code_report.txt
-test_code_report.txt
+*test_code_report.txt
+*coverage*
+
+tests/*.nii*
+tests/*.json*
+tests/*.tsv*
 
 # exclude report from check_my_code
 check_my_code_report.txt

README.md

@@ -1,5 +1,20 @@
-[![All Contributors](https://img.shields.io/badge/all_contributors-3-orange.svg?style=flat-square)](#contributors-) [![Build Status](https://travis-ci.com/cpp-lln-lab/CPP_BIDS.svg?branch=master)](https://travis-ci.com/cpp-lln-lab/CPP_BIDS)
+**Unit tests and coverage**
 
+[![](https://img.shields.io/badge/Octave-CI-blue?logo=Octave&logoColor=white)](https://github.com/cpp-lln-lab/CPP_BIDS/actions)
+![](https://github.com/cpp-lln-lab/CPP_BIDS/workflows/CI/badge.svg) 
+
+[![codecov](https://codecov.io/gh/cpp-lln-lab/CPP_BIDS/branch/master/graph/badge.svg)](https://codecov.io/gh/cpp-lln-lab/CPP_BIDS)
+
+**BIDS validator and linter**
+
+[![Build Status](https://travis-ci.com/cpp-lln-lab/CPP_BIDS.svg?branch=master)](https://travis-ci.com/cpp-lln-lab/CPP_BIDS)
+
+**Contributors**
+
+[![All Contributors](https://img.shields.io/badge/all_contributors-3-orange.svg?style=flat-square)](#contributors-) 
+
+---
+ 
 # CPP_BIDS
 
 <!-- TOC -->
@@ -15,9 +30,11 @@
     - [saveEventsFile](#saveeventsfile)
     - [checkCFG](#checkcfg)
   - [CFG content](#cfg-content)
+    - [createBoldJson](#createboldjson)
   - [How to install](#how-to-install)
     - [Download with git](#download-with-git)
     - [Add as a submodule](#add-as-a-submodule)
+      - [Example for submodule usage](#example-for-submodule-usage)
     - [Direct download](#direct-download)
   - [Contributing](#contributing)
     - [Guidestyle](#guidestyle)
@@ -36,7 +53,7 @@ Subjects, session and run number labels will be numbers with zero padding up to
 
 A session folder will ALWAYS be created even if not requested (default will be `ses-001`).
 
-Task labels will be printed in camelCase in the filenames. 
+Task labels will be printed in camelCase in the filenames.
 
 Time stamps are added directly in the filename by adding a suffix `_date-YYYYMMDDHHMM` which makes the file name non-BIDS compliant. This was added to prevent overwriting files in case a certain run needs to be done a second time because of a crash (Some of us are paranoid about keeping even cancelled runs during my experiments). This suffix should be removed to make the data set BIDS compliant. See `convertSourceToRaw.m` for more details.
 
@@ -56,14 +73,14 @@ sub-090/ses-003/sub-090_ses-003_task-auditoryTask_run-023_events_date-2020072915
 cfg.outputDir = fullfile(pwd, '..', 'output');
 
 % define the name of the task
-cfg.task = 'test task';
+cfg.task.name = 'test task';
 
 % can use the userInputs function to collect subject info
 % cfg = userInputs;
 
 % or declare it directly
-cfg.subjectNb = 1;
-cfg.runNb = 1;
+cfg.subject.subjectNb = 1;
+cfg.subject.runNb = 1;
 
 % by default we assume you are running things on a behavioral PC with no eyetracker
 % cfg.eyeTracker = false;
@@ -109,9 +126,9 @@ saveEventsFile('close', cfg, logFile);
 If you want to save more complex events.tsv file you can save several columns at once.
 
 ```matlab
-cfg.subjectNb = 1;
-cfg.runNb = 1;
-cfg.task = 'testtask';
+cfg.subject.subjectNb = 1;
+cfg.subject.runNb = 1;
+cfg.task.name = 'testtask';
 cfg.outputDir = outputDir;
 
 cfg.testingDevice = 'mri';
@@ -206,7 +223,7 @@ For the moment the date of acquisition is appended to the filename
 
 Function to save output files for events that will be BIDS compliant.
 
-If the user DOES NOT provide `onset`, `trial_type`, this events will be skipped. `duration` will be set to "NaN" if 
+If the user DOES NOT provide `onset`, `trial_type`, this events will be skipped. `duration` will be set to "NaN" if
 no value is provided.
 
 ### checkCFG
@@ -216,7 +233,7 @@ Check that we have all the fields that we need in the experiment parameters.
 ## CFG content
 
 ```matlab
-%% Can be modified by users 
+%% Can be modified by users
 % but their effect might only be effective after running
 % checkCFG
 
@@ -281,6 +298,26 @@ cfg.fileName.datasetDescription
 
 ```
 
+### createBoldJson
+
+```
+createBoldJson(cfg)
+```
+
+This function creates a very light-weight version of the side-car JSON file for a BOLD functional run.
+
+This will only contain the minimum BIDS requirement and will likely be less complete than the info you could from DICOM conversion.
+
+If you put the following line at the end of your experiment script, it will dump the content of the `extraInfo` structure in the json file. 
+
+```
+createBoldJson(cfg, extraInfo)
+```
+
+This allows to add all the parameters that you used to run your experiment in a human readable format: so that when you write your methods sections 2 years later ("the reviewer asked me for the size of my fixation cross... FML"), the info you used WHEN you ran the experiment is saved in an easily accessible text format. For the love of the flying spaghetti monster do not save all your parameters in a `.mat` file: think of the case when you won't have matlab or octave installed on a computer (plus not everyone uses those).
+
+Also to reading your experiment parameters, you won't have to read it from the `setParameters.m` file and wonder if those might have been modified when running the experiment and you did not commit and tagged that change with git.
+
 ## How to install
 
 ### Download with git
@@ -290,7 +327,7 @@ cd fullpath_to_directory_where_to_install
 # use git to download the code
 git clone https://github.com/cpp-lln-lab/CPP_BIDS.git
 # move into the folder you have just created
-cd CPP_PTB
+cd CPP_BIDS
 # add the src folder to the matlab path and save the path
 matlab -nojvm -nosplash -r "addpath(fullfile(pwd, 'src')); savepath ();"
 ```
@@ -316,18 +353,37 @@ cd fullpath_to_directory_where_to_install
 # use git to download the code
 git submodule add https://github.com/cpp-lln-lab/CPP_BIDS.git
 # move into the folder you have just created
-cd CPP_PTB
+cd CPP_BIDS
 # add the src folder to the matlab path and save the path
 matlab -nojvm -nosplash -r "addpath(fullfile(pwd, 'src'))"
 ```
 
-To get the latest commit you then need to update the submodule with the information 
+To get the latest commit you then need to update the submodule with the information
 on its remote repository and then merge those locally.
 ```bash
 git submodule update --remote --merge
 ```
 
-Remember that updates to submodules need to be commited as well. 
+Remember that updates to submodules need to be committed as well.
+
+#### Example for submodule usage
+
+So say you want to clone a repo that has some nested submodules, then you would type this to get the content of all the submodules at once (here with assumption that you want to clone my experiment repo):
+```bash
+# clone the repo
+git clone https://github.com/user_name/myExperiment.git
+
+# go into the directory
+cd myExperiment
+
+# initialize and get the content of the first level of submodules  (e.g. CPP_PTB and CPP_BIDS)
+git submodule init
+git submodule update
+
+# get the nested submodules JSONio and BIDS-matlab for CPP_BIDS
+git submodule foreach --recursive 'git submodule init'
+git submodule foreach --recursive 'git submodule update'
+```
 
 **TO DO**
 <!-- Submodules

dummyData/CHANGES manualTests/dummyData/CHANGESdummyData/CHANGES renamed to manualTests/dummyData/CHANGES

@@ -0,0 +1,10 @@
+{
+    "Instructions": "",
+    "RepetitionTime": [],
+    "SliceTiming": [],
+    "TaskDescription": "",
+    "TaskName": "testtask",
+    "extraInfo": {
+        "nestedExtraInfo": "something extra"
+    }
+}

dummyData/README manualTests/dummyData/READMEdummyData/README renamed to manualTests/dummyData/README

@@ -0,0 +1,50 @@
+function test_suite = test_createBoldJson %#ok<*STOUT>
+    try % assignment of 'localfunctions' is necessary in Matlab >= 2016
+        test_functions = localfunctions(); %#ok<*NASGU>
+    catch % no problem; early Matlab versions can use initTestSuite fine
+    end
+    initTestSuite;
+end
+
+function test_createBoldJsonExtra()
+
+    outputDir = fullfile(fileparts(mfilename('fullpath')), 'output');
+
+    %% set up
+
+    cfg.verbose = false;
+
+    cfg.subject.subjectNb = 1;
+    cfg.subject.runNb = 1;
+
+    cfg.task.name = 'testtask';
+
+    cfg.dir.output = outputDir;
+
+    cfg.testingDevice = 'mri';
+
+    cfg = createFilename(cfg);
+
+    logFile = saveEventsFile('init', cfg); %#ok<*NASGU>
+
+    extraInfo = struct('extraInfo', struct('nestedExtraInfo', 'something extra'));
+
+    createBoldJson(cfg, extraInfo);
+
+    %% check content
+    fileName = strrep(cfg.fileName.events, '_events', '_bold');
+    fileName = strrep(fileName, '.tsv', '.json');
+
+    actualStruct = bids.util.jsondecode(fullfile( ...
+        cfg.dir.outputSubject, ...
+        cfg.fileName.modality, ...
+        fileName));
+
+    % data to test against
+    expectedStruct = bids.util.jsondecode( ...
+        fullfile(pwd, 'testData', 'extra_bold.json'));
+
+    % test
+    assertEqual(expectedStruct, actualStruct);
+
+end

dummyData/dummyData.nii.gz manualTests/dummyData/dummyData.nii.gzdummyData/dummyData.nii.gz renamed to manualTests/dummyData/dummyData.nii.gz

@@ -7,38 +7,36 @@ function test_makeRawDataset()
     end
 
     %% set up
-
-    %%% set up
-
-    cfg.subject.subjectNb = 1;
-    cfg.subject.runNb = 1;
-
-    cfg.task.name = 'testtask';
-
     cfg.dir.output = outputDir;
 
     cfg.bids.datasetDescription.Name = 'dummy';
     cfg.bids.datasetDescription.BIDSVersion = '1.0.0';
     cfg.bids.datasetDescription.Authors = {'Jane Doe', 'John Doe'};
 
-    cfg.bids.mri.RepetitionTime = 1.56;
-
     cfg.testingDevice = 'mri';
 
+    %% MRI task data
+    cfg.mri.repetitionTime = 1.56;
+
+    cfg.subject.subjectNb = 1;
+    cfg.subject.runNb = 1;
+
+    cfg.task.name = 'testtask';
+
     logFile.extraColumns.Speed.length = 1;
     logFile.extraColumns.LHL24.length = 3;
     logFile.extraColumns.is_Fixation.length = 1;
 
-    %%% do stuff
-
     cfg = createFilename(cfg);
 
-    % create the events file and header
-    logFile = saveEventsFile('open', cfg, logFile);
+    extraInfo = struct('extraInfo', struct('nestedExtraInfo', 'something extra'));
+    createBoldJson(cfg, extraInfo);
 
-    createBoldJson(cfg);
     createDatasetDescription(cfg);
 
+    % create the events file and header
+    logFile = saveEventsFile('open', cfg, logFile);
+
     % ROW 2: normal events : all info is there
     logFile(1, 1).onset = 2;
     logFile(end, 1).trial_type = 'MotionUp';
@@ -71,14 +69,67 @@ function test_makeRawDataset()
     % close the file
     saveEventsFile('close', cfg, logFile);
 
+    % add dummy stim data
+    stimLogFile = saveEventsFile('open_stim', cfg, logFile);
+    for i = 1:100
+        stimLogFile(i, 1).onset = cfg.mri.repetitionTime * i;
+        stimLogFile(i, 1).trial_type = 'test';
+        stimLogFile(i, 1).duration = 1;
+        stimLogFile(i, 1).Speed = rand(1);
+        stimLogFile(i, 1).is_Fixation = rand > 0.5;
+        stimLogFile(i, 1).LHL24 = randn(1, 3);
+    end
+    saveEventsFile('save', cfg, stimLogFile);
+    saveEventsFile('close', cfg, stimLogFile);
+
     % add dummy functional data
     funcDir = fullfile(cfg.dir.output, 'source', 'sub-001', 'ses-001', 'func');
     boldFilename = 'sub-001_ses-001_task-testtask_run-001_bold.nii.gz';
+
     copyfile( ...
-        fullfile('..', 'dummyData', 'dummyData.nii.gz'), ...
+        fullfile('dummyData', 'dummyData.nii.gz'), ...
         fullfile(funcDir, boldFilename));
 
-    %%
+    %% MRI bold rest data and fancy suffixes
+    clear cfg;
+
+    cfg.dir.output = outputDir;
+
+    cfg.testingDevice = 'mri';
+
+    cfg.subject.subjectNb = 2;
+    cfg.subject.sessionNb = 3;
+    cfg.subject.runNb = 4;
+
+    % deal with MRI suffixes
+    cfg.mri.reconstruction = 'fast recon';
+    cfg.mri.contrastEnhancement = 'test';
+    cfg.mri.phaseEncodingDirection = 'y pos';
+    cfg.mri.echo = '1';
+    cfg.mri.acquisition = ' new tYpe';
+    cfg.mri.repetitionTime = 1.56;
+
+    cfg.task.name = 'rest';
+
+    cfg = createFilename(cfg);
 
+    createBoldJson(cfg);
+
+    %% add dummy functional data
+    funcDir = fullfile(cfg.dir.output, 'source', 'sub-002', 'ses-003', 'func');
+    boldFilename = ['sub-002_ses-003_task-rest',  ...
+        '_acq-newTYpe_ce-test_dir-yPos_rec-fastRecon', ...
+        '_run-004_echo-1_bold.nii.gz'];
+
+    copyfile( ...
+        fullfile('dummyData', 'dummyData.nii.gz'), ...
+        fullfile(funcDir, boldFilename));
+
+    %% actually do the conversion of the source data thus created
+    clear;
+
+    outputDir = fullfile(fileparts(mfilename('fullpath')), 'output');
+    cfg.dir.output = outputDir;
     convertSourceToRaw(cfg);
+
 end