Ensure timestamp checks are consistent

[elichad]

We have checks that apply to startTime, endTime, and maybe other datetime properties. We should make sure they follow a consistent pattern for the timestamps.

Sparked by discussion on #40:

Do we need to worry about milliseconds / fractional seconds? Just wondering as auto-generated metadata / libraries may output this and therefore this will fail. Not sure its specified in RFC 3339

Secondly, offsets without colons will fail. e.g from Wikipedia: "...Cairo will be "+02:00","+0200", or "+02"."

Originally posted by @alexhambley in #40 (comment)

I think we should add the fractional seconds (with ',' or '.' as the delimiter).

I think we should retain ':' delimiter in the timezones, I don't know if we should allow only HH though - I couldn't see easily if this was allowed in the strict definition.

Maybe we need an example in our error messages of the format we accept?

Originally posted by @douglowe in #40 (comment)

[elichad]

We could create a reusable NodeShape that defines the pattern just once, and use sh:node in other shapes to reference it. See example

[douglowe]

I've prototyped some reusable NodeShapes - see the attached PR for these.

An issue I've encountered is that the report layout is changed by using a modular library. I've included an example of this below.

I've setup both the SignOff and the WorkflowExection tests to use the new EndTimeStamp object. The SignOff test also uses the new StartTimeStamp object, and WorkflowExection is also using the original end time test.

Our report has violation reports for the SignOff and the WorkflowExection stages, but these are rather generic. There is now also the violation reports for EndTimeStamp and StartTimeStamp. These contain the full error message, and the IDs of the objects where these occur, but are not nested within the relevant phase checks.

I have tried creating PropertyShape rather than NodeShape objects for these tests. However I get an iteration error (TypeError: argument of type 'NoneType' is not iterable). This is a python error which I think this is due to the way the validator iterates over the SHACL shapes, and I guess is related method by which we get the numbering which we have for the different tests. Without a major rework of the core validator code I don't think we can address this error.

So - the question here is - is this an acceptable way to present the validator results? Or do we think this is going to be too confusing for end users.

                                                                                                                        
  The following requirements have not meet:                                                                             
                                                                                                                        
                                                                                      [profile: Five Safes RO-Crate 0.4]
     [ five-safes-crate-0.4_13 ]: SignOff                                                                               
                                                                                                                        
      Sign Off phase                                                                                                    
                                                                                                                        
          Failed checks                                                                                                 
                                                                                                                        
       [ five-safes-crate-0.4_13.0 ] Check SignOff:                                                                     
                                      Sign Off phase                                                                    
         Detected issues                                                                                                
         -                                                                                                              
         [Violation"file://tests/data/crates/valid/five-safes-crate-result/#signoff-3b741265-cfef-49ea-8138-a2f         
         a149bf2f0"  on <./#signoff-3b741265-cfef-49ea-8138-a2fa149bf2f0>]: Value must conform to every Shape           
         in ('<https://github.com/eScienceLab/rocrate-validator/profiles/five-safes-crate/EndTimeStamp>',               
         '<https://github.com/eScienceLab/rocrate-validator/profiles/five-safes-crate/StartTimeStamp>'). See            
         details for more information.                                                                                  
                                                                                                                        
                                                                                                                        
                                                                                                                        
                                                                                      [profile: Five Safes RO-Crate 0.4]
     [ five-safes-crate-0.4_23 ]: EndTimeStamp                                                                          
                                                                                                                        
      Profile Requirement EndTimeStamp                                                                                  
                                                                                                                        
          Failed checks                                                                                                 
                                                                                                                        
       [ five-safes-crate-0.4_23.1 ] EndTimeStamp:                                                                      
                                      End Time Check                                                                    
         Detected issues                                                                                                
         - [Violation of http://schema.org/endTime="also dodgy"  on                                                     
         <./#signoff-3b741265-cfef-49ea-8138-a2fa149bf2f0>]: endTime property MUST follow ISO-8601 syntax. e.g.         
         2023-04-19T10:15:12+01:00                                                                                      
         - [Violation of http://schema.org/endTime="broken string"  on                                                  
         <./#query-37252371-c937-43bd-a0a7-3680b48c0538>]: endTime property MUST follow ISO-8601 syntax. e.g.           
         2023-04-19T10:15:12+01:00                                                                                      
                                                                                                                        
                                                                                                                        
                                                                                                                        
                                                                                      [profile: Five Safes RO-Crate 0.4]
     [ five-safes-crate-0.4_24 ]: StartTimeStamp                                                                        
                                                                                                                        
      Profile Requirement StartTimeStamp                                                                                
                                                                                                                        
          Failed checks                                                                                                 
                                                                                                                        
       [ five-safes-crate-0.4_24.1 ] StartTimeStamp:                                                                    
                                      Start Time Check                                                                  
         Detected issues                                                                                                
         - [Violation of http://schema.org/startTime="dodgy"  on                                                        
         <./#signoff-3b741265-cfef-49ea-8138-a2fa149bf2f0>]: startTime property MUST follow ISO-8601 syntax.            
         e.g. 2023-04-19T10:15:12+01:00                                                                                 
                                                                                                                        
                                                                                                                        
                                                                                                                        
                                                                                      [profile: Five Safes RO-Crate 0.4]
     [ five-safes-crate-0.4_3 ]: WorkflowExecution                                                                      
                                                                                                                        
      The endTime of the workflow execution object MUST follow the RFC 3339 standard                                    
      (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                                                   
                                                                                                                        
          Failed checks                                                                                                 
                                                                                                                        
       [ five-safes-crate-0.4_3.0 ] Check WorkflowExecution:                                                            
                                     The endTime of the workflow execution object MUST follow the RFC 3339 standard     
                                     (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                    
         Detected issues                                                                                                
         -                                                                                                              
         [Violation"file://tests/data/crates/valid/five-safes-crate-result/#query-37252371-c937-43bd-a0a7-3680b         
         48c0538"  on <./#query-37252371-c937-43bd-a0a7-3680b48c0538>]: Value does not conform to Shape                 
         <https://github.com/eScienceLab/rocrate-validator/profiles/five-safes-crate/EndTimeStamp>. See details         
         for more information.                                                                                          
       [ five-safes-crate-0.4_3.1 ] EndTime:                                                                            
                                     Check the property "EndTime" of the entity "WorkflowExecution"                     
         Detected issues                                                                                                
         - [Violation of http://schema.org/endTime="broken string"  on                                                  
         <./#query-37252371-c937-43bd-a0a7-3680b48c0538>]: The endTime of the workflow execution object MUST            
         follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                   
                                                                                                                        
                                                                                                                       

[elichad]

I came up with an alternative approach using sh:targetObjectsOf, which selects all objects of the listed predicates across all entities.

To test it I had to rework some of the existing tests and remove the existing checks, so I ended up making #57.

five-safes-crate:TimeStampFormat
    a sh:NodeShape ;
    sh:name "Timestamp Format" ;
    sh:description "Timestamps MUST follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm))." ;
    sh:targetObjectsOf schema:startTime, schema:endTime;
    sh:pattern "^[0-9]{4}-[0-9]{2}-[0-9]{2}[Tt][0-9]{2}:[0-9]{2}:[0-9]{2}([.|,][0-9]+)?(Z|z|[+-][0-9]{2}:[0-9]{2})$" ;
    sh:severity sh:Violation ;
    sh:message "All `startTime` and `endTime` values MUST follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm))." .

This generates more straightforward error messages (e.g. with the timestamp "01 Jan 2026 10:00") :

     [ five-safes-crate-0.4_23 ]: Timestamp Format                                                                                    
                                                                                                                                      
      Timestamps MUST follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                    
                                                                                                                                      
          Failed checks                                                                                                               
                                                                                                                                      
       [ five-safes-crate-0.4_23.0 ] Timestamp Format:                                                                                
                                      Timestamps MUST follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).    
         Detected issues                                                                                                              
         - [Violation"01 Jan 2026 10:00"  on <01 Jan 2026 10:00>]: All `startTime` and `endTime` values MUST follow the RFC           
         3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                                                
                                                                              

[douglowe]

This does look interesting - but I think it misses vital information to help users identify the source of their problem.

I've expanded on your solution in #59, by using sh:targetSubjectsOf instead of sh:targetObjectsOf, and including 2 optional properties, one for startTime and the other for endTime (separated because SHACL needs absolute paths in these properties).

This produces reports such as:

     [ five-safes-crate-0.4_25 ]: Timestamp Format                                                                      
                                                                                                                        
      Timestamps MUST follow the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                      
                                                                                                                        
          Failed checks                                                                                                 
                                                                                                                        
       [ five-safes-crate-0.4_25.1 ] End TimeStamp:                                                                     
                                      Check the property "End TimeStamp" of the entity "Timestamp Format"               
         Detected issues                                                                                                
         - [Violation of http://schema.org/endTime="dodgy endtime"  on                                                  
         <./#check-f33fe90c-0c22-4c72-b299-de509028410e>]: All `startTime` and `endTime` values MUST follow the         
         RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                              
         - [Violation of http://schema.org/endTime="invalid time"  on                                                   
         <./#query-37252371-c937-43bd-a0a7-3680b48c0538>]: All `startTime` and `endTime` values MUST follow the         
         RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                              
       [ five-safes-crate-0.4_25.2 ] Start TimeStamp:                                                                   
                                      Check the property "Start TimeStamp" of the entity "Timestamp Format"             
         Detected issues                                                                                                
         - [Violation of http://schema.org/startTime="another wrong starttime"  on                                      
         <./#check-f33fe90c-0c22-4c72-b299-de509028410e>]: All `startTime` and `endTime` values MUST follow the         
         RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                              
         - [Violation of http://schema.org/startTime="silly starttime"  on                                              
         <./#validate-1146f640-819e-4c86-b029-b763a0040896>]: All `startTime` and `endTime` values MUST follow          
         the RFC 3339 standard (YYYY-MM-DD'T'hh:mm:ss[.fraction](Z | ±hh:mm)).                                          
                                                                                                                        

Here the id of each offending object is included, rather than repeating the date string again.

[douglowe]

We've adopted the solution in PR #57 - this is more elegant than my original suggested solution.