@@ -6,20 +6,18 @@ DataJoint uses global state (`dj.config`, `dj.conn()`) that is not thread-safe.
66
77## Solution
88
9- Introduce ** context** objects that encapsulate config and connection. The ` dj ` module itself is the singleton (legacy) context. New isolated contexts are created with ` dj.new() ` .
9+ Introduce ** context** objects that encapsulate config and connection. The ` dj ` module provides the singleton (legacy) context. New isolated contexts are created with ` dj.new() ` .
1010
1111## API
1212
1313### Legacy API (singleton context)
1414
15- The ` dj ` module acts as the default singleton context:
16-
1715``` python
1816import datajoint as dj
1917
2018dj.config.safemode = False
2119dj.conn(host = " localhost" , user = " u" , password = " p" )
22- schema = dj.Schema(" my_schema" ) # Uses dj's connection
20+ schema = dj.Schema(" my_schema" )
2321
2422@schema
2523class Mouse (dj .Manual ):
@@ -28,39 +26,34 @@ class Mouse(dj.Manual):
2826
2927### New API (isolated context)
3028
31- Create isolated contexts with ` dj.new() ` :
32-
3329``` python
3430import datajoint as dj
3531
36- ctx = dj.new() # New context with its own config copy
32+ ctx = dj.new(
33+ host = " localhost" ,
34+ user = " user" ,
35+ password = " password" ,
36+ )
3737ctx.config.safemode = False
38- ctx.connect(host = " localhost" , user = " u" , password = " p" )
39- schema = ctx.Schema(" my_schema" ) # Uses ctx's connection
38+ schema = ctx.Schema(" my_schema" )
4039
4140@schema
42- class Mouse (ctx .Manual ):
41+ class Mouse (dj .Manual ):
4342 definition = " ..."
4443```
4544
4645### Context structure
4746
48- Each context has:
49- - ** One config** - copy of settings at creation time
50- - ** One connection** - established via ` ctx.connect() `
51- - ** Schema factory** - ` ctx.Schema() ` auto-uses context's connection
52- - ** Table base classes** - ` ctx.Manual ` , ` ctx.Lookup ` , ` ctx.Imported ` , ` ctx.Computed ` , ` ctx.Part `
47+ Each context exposes only:
48+ - ` ctx.config ` - Config instance (copy of ` dj.config ` at creation)
49+ - ` ctx.connection ` - Connection (created at context construction)
50+ - ` ctx.Schema() ` - Schema factory using context's connection
5351
5452``` python
55- ctx = dj.new()
56- ctx.config # Config instance (copy of dj.config at creation)
57- ctx.connect(... ) # Establish connection
58- ctx.Schema(... ) # Create schema using ctx's connection
59- ctx.Manual # Base class for manual tables
60- ctx.Lookup # Base class for lookup tables
61- ctx.Imported # Base class for imported tables
62- ctx.Computed # Base class for computed tables
63- ctx.Part # Base class for part tables
53+ ctx = dj.new(host = " localhost" , user = " u" , password = " p" )
54+ ctx.config # Config instance
55+ ctx.connection # Connection instance
56+ ctx.Schema(" name" ) # Creates schema using ctx.connection
6457```
6558
6659### Thread-safe mode
@@ -72,79 +65,71 @@ export DJ_THREAD_SAFE=true
7265When ` thread_safe=True ` :
7366- ` dj.conn() ` raises ` ThreadSafetyError `
7467- ` dj.Schema() ` raises ` ThreadSafetyError `
75- - ` dj.config ` is read- only
68+ - ` dj.config ` only allows access to ` thread_safe ` (all other access raises ` ThreadSafetyError ` )
7669- ` dj.new() ` works - isolated contexts are always allowed
7770
7871``` python
7972# thread_safe=True
8073
81- dj.Schema( " name " ) # ThreadSafetyError
82- dj.conn() # ThreadSafetyError
74+ dj.config.thread_safe # OK - allowed
75+ dj.config.safemode # ThreadSafetyError
8376dj.config.safemode = False # ThreadSafetyError
77+ dj.conn() # ThreadSafetyError
78+ dj.Schema(" name" ) # ThreadSafetyError
8479
85- ctx = dj.new() # OK - isolated context
86- ctx.config.safemode = False # OK - context's own config
87- ctx.connect(... ) # OK
88- ctx.Schema(" name" ) # OK
80+ ctx = dj.new(host = " h" , user = " u" , password = " p" ) # OK
81+ ctx.config.safemode = False # OK
82+ ctx.Schema(" name" ) # OK
8983```
9084
9185## Behavior Summary
9286
9387| Operation | ` thread_safe=False ` | ` thread_safe=True ` |
9488| -----------| --------------------| --------------------|
95- | ` dj.config ` read | Works | Works |
96- | ` dj.config ` write | Works | ` ThreadSafetyError ` |
89+ | ` dj.config.thread_safe ` | Works | Works |
90+ | ` dj.config.* ` (other) | Works | ` ThreadSafetyError ` |
9791| ` dj.conn() ` | Works | ` ThreadSafetyError ` |
9892| ` dj.Schema() ` | Works | ` ThreadSafetyError ` |
9993| ` dj.new() ` | Works | Works |
100- | ` ctx.config ` read/write | Works | Works |
101- | ` ctx.connect() ` | Works | Works |
94+ | ` ctx.config.* ` | Works | Works |
95+ | ` ctx.connection ` | Works | Works |
10296| ` ctx.Schema() ` | Works | Works |
10397
104- ## Context Lifecycle
98+ ## Usage Example
10599
106100``` python
107- # Create context
108- ctx = dj.new()
109-
110- # Configure
111- ctx.config.database.host = " localhost"
112- ctx.config.safemode = False
113- ctx.config.stores = {... }
101+ import datajoint as dj
114102
115- # Connect
116- ctx.connect (
117- host = " localhost" , # Or use ctx.config.database.host
103+ # Create isolated context
104+ ctx = dj.new (
105+ host = " localhost" ,
118106 user = " user" ,
119107 password = " password" ,
120108)
121109
122- # Use
110+ # Configure
111+ ctx.config.safemode = False
112+ ctx.config.stores = {" raw" : {" protocol" : " file" , " location" : " /data" }}
113+
114+ # Create schema
123115schema = ctx.Schema(" my_schema" )
124116
125117@schema
126- class Mouse (ctx .Manual ):
118+ class Mouse (dj .Manual ):
127119 definition = """
128120 mouse_id: int
129121 """
130122
131- Mouse().insert1({" mouse_id" : 1 })
132-
133- # Cleanup (optional - closes connection)
134- ctx.close()
135- ```
136-
137- ## Legacy Compatibility
138-
139- The singleton ` dj ` context works exactly as before:
140-
141- ``` python
142- # These are equivalent:
143- dj.conn() # Singleton connection
144- dj.config # Singleton config
145- dj.Schema(" name" ) # Uses singleton connection
123+ @schema
124+ class Session (dj .Manual ):
125+ definition = """
126+ -> Mouse
127+ session_date: date
128+ """
146129
147- # Internally, dj module delegates to singleton context
130+ # Use tables
131+ Mouse().insert1({" mouse_id" : 1 })
132+ Mouse().delete() # Uses ctx.config.safemode
148133```
149134
150135## Implementation
@@ -153,68 +138,46 @@ dj.Schema("name") # Uses singleton connection
153138
154139``` python
155140class Context :
156- def __init__ (self , config : Config):
157- self .config = config
158- self ._connection = None
159-
160- def connect (self , host , user , password , ...):
161- self ._connection = Connection(... )
162- self ._connection.config = self .config
163-
164- def conn (self ):
165- return self ._connection
166-
167- def Schema (self , name , ...):
168- return Schema(name, connection = self ._connection, ... )
169-
170- # Table base classes that reference this context
171- @ property
172- def Manual (self ): ...
173- @ property
174- def Lookup (self ): ...
175- # etc.
141+ def __init__ (self , host , user , password , port = 3306 , ...):
142+ self .config = copy(dj.config) # Independent config copy
143+ self .connection = Connection(host, user, password, port, ... )
144+ self .connection._config = self .config # Link config to connection
145+
146+ def Schema (self , name , ** kwargs ):
147+ return Schema(name, connection = self .connection, ** kwargs)
176148```
177149
178150### 2. Add dj.new()
179151
180152``` python
181- def new () -> Context:
153+ def new (host , user , password , ** kwargs ) -> Context:
182154 """ Create a new isolated context with its own config and connection."""
183- config_copy = copy(config) # Copy current global config
184- return Context(config_copy)
155+ return Context(host, user, password, ** kwargs)
185156```
186157
187- ### 3. Make dj module act as singleton context
158+ ### 3. Add thread_safe guards
159+
160+ In ` dj.config ` :
161+ - Allow read/write of ` thread_safe ` always
162+ - When ` thread_safe=True ` , block all other attribute access
188163
189164``` python
190- # In datajoint/__init__.py
191- _singleton_context = Context(config)
192-
193- def conn (...):
194- if config.thread_safe:
195- raise ThreadSafetyError(... )
196- return _singleton_context.conn(... )
197-
198- def Schema (...):
199- if config.thread_safe:
200- raise ThreadSafetyError(... )
201- return _singleton_context.Schema(... )
165+ def __getattr__ (self , name ):
166+ if name == " thread_safe" :
167+ return self ._thread_safe
168+ if self ._thread_safe:
169+ raise ThreadSafetyError(" Global config is inaccessible in thread-safe mode." )
170+ # ... normal access
202171```
203172
204- ### 4. Add thread_safe guards
205-
206- - ` dj.conn() ` : Raise ` ThreadSafetyError ` when ` thread_safe=True `
207- - ` dj.Schema() ` : Raise ` ThreadSafetyError ` when ` thread_safe=True `
208- - ` dj.config ` writes: Raise ` ThreadSafetyError ` when ` thread_safe=True `
209-
210- ### 5. Refactor internal code
173+ ### 4. Refactor internal code
211174
212- All internal code uses ` self.connection.config ` instead of global ` config ` :
213- - Tables access config via ` self.connection.config `
214- - Connection has reference to its context's config
175+ All internal code uses ` self.connection._config ` instead of global ` config ` :
176+ - Tables access config via ` self.connection._config `
177+ - This works uniformly for both singleton and isolated contexts
215178
216179## Error Messages
217180
218- - ` dj.conn() ` : ` "dj.conn() is disabled in thread-safe mode. Use ctx = dj.new() to create an isolated context ." `
219- - ` dj.Schema () ` : ` "dj.Schema () is disabled in thread-safe mode. Use ctx = dj.new() to create an isolated context." `
220- - ` dj.config ` write : ` "Global config is read-only in thread-safe mode. Use ctx = dj.new() for isolated config ." `
181+ - ` dj.config.* ` : ` "Global config is inaccessible in thread-safe mode. Use ctx = dj.new(...) for isolated config ." `
182+ - ` dj.conn () ` : ` "dj.conn () is disabled in thread-safe mode. Use ctx = dj.new(... ) to create an isolated context." `
183+ - ` dj.Schema() ` : ` "dj.Schema() is disabled in thread-safe mode. Use ctx = dj.new(...) to create an isolated context ." `
0 commit comments