Skip to content

Commit f92af1c

Browse files
docs: Simplify context - only config, connection, Schema
- ctx exposes only: config, connection, Schema() - Connection created at context construction via dj.new() - Tables still use dj.Manual, dj.Lookup as base classes - thread_safe=True: dj.config only allows thread_safe access Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent bb7adfd commit f92af1c

File tree

1 file changed

+76
-113
lines changed

1 file changed

+76
-113
lines changed

docs/design/thread-safe-mode.md

Lines changed: 76 additions & 113 deletions
Original file line numberDiff line numberDiff line change
@@ -6,20 +6,18 @@ DataJoint uses global state (`dj.config`, `dj.conn()`) that is not thread-safe.
66

77
## Solution
88

9-
Introduce **context** objects that encapsulate config and connection. The `dj` module itself is the singleton (legacy) context. New isolated contexts are created with `dj.new()`.
9+
Introduce **context** objects that encapsulate config and connection. The `dj` module provides the singleton (legacy) context. New isolated contexts are created with `dj.new()`.
1010

1111
## API
1212

1313
### Legacy API (singleton context)
1414

15-
The `dj` module acts as the default singleton context:
16-
1715
```python
1816
import datajoint as dj
1917

2018
dj.config.safemode = False
2119
dj.conn(host="localhost", user="u", password="p")
22-
schema = dj.Schema("my_schema") # Uses dj's connection
20+
schema = dj.Schema("my_schema")
2321

2422
@schema
2523
class Mouse(dj.Manual):
@@ -28,39 +26,34 @@ class Mouse(dj.Manual):
2826

2927
### New API (isolated context)
3028

31-
Create isolated contexts with `dj.new()`:
32-
3329
```python
3430
import datajoint as dj
3531

36-
ctx = dj.new() # New context with its own config copy
32+
ctx = dj.new(
33+
host="localhost",
34+
user="user",
35+
password="password",
36+
)
3737
ctx.config.safemode = False
38-
ctx.connect(host="localhost", user="u", password="p")
39-
schema = ctx.Schema("my_schema") # Uses ctx's connection
38+
schema = ctx.Schema("my_schema")
4039

4140
@schema
42-
class Mouse(ctx.Manual):
41+
class Mouse(dj.Manual):
4342
definition = "..."
4443
```
4544

4645
### Context structure
4746

48-
Each context has:
49-
- **One config** - copy of settings at creation time
50-
- **One connection** - established via `ctx.connect()`
51-
- **Schema factory** - `ctx.Schema()` auto-uses context's connection
52-
- **Table base classes** - `ctx.Manual`, `ctx.Lookup`, `ctx.Imported`, `ctx.Computed`, `ctx.Part`
47+
Each context exposes only:
48+
- `ctx.config` - Config instance (copy of `dj.config` at creation)
49+
- `ctx.connection` - Connection (created at context construction)
50+
- `ctx.Schema()` - Schema factory using context's connection
5351

5452
```python
55-
ctx = dj.new()
56-
ctx.config # Config instance (copy of dj.config at creation)
57-
ctx.connect(...) # Establish connection
58-
ctx.Schema(...) # Create schema using ctx's connection
59-
ctx.Manual # Base class for manual tables
60-
ctx.Lookup # Base class for lookup tables
61-
ctx.Imported # Base class for imported tables
62-
ctx.Computed # Base class for computed tables
63-
ctx.Part # Base class for part tables
53+
ctx = dj.new(host="localhost", user="u", password="p")
54+
ctx.config # Config instance
55+
ctx.connection # Connection instance
56+
ctx.Schema("name") # Creates schema using ctx.connection
6457
```
6558

6659
### Thread-safe mode
@@ -72,79 +65,71 @@ export DJ_THREAD_SAFE=true
7265
When `thread_safe=True`:
7366
- `dj.conn()` raises `ThreadSafetyError`
7467
- `dj.Schema()` raises `ThreadSafetyError`
75-
- `dj.config` is read-only
68+
- `dj.config` only allows access to `thread_safe` (all other access raises `ThreadSafetyError`)
7669
- `dj.new()` works - isolated contexts are always allowed
7770

7871
```python
7972
# thread_safe=True
8073

81-
dj.Schema("name") # ThreadSafetyError
82-
dj.conn() # ThreadSafetyError
74+
dj.config.thread_safe # OK - allowed
75+
dj.config.safemode # ThreadSafetyError
8376
dj.config.safemode = False # ThreadSafetyError
77+
dj.conn() # ThreadSafetyError
78+
dj.Schema("name") # ThreadSafetyError
8479

85-
ctx = dj.new() # OK - isolated context
86-
ctx.config.safemode = False # OK - context's own config
87-
ctx.connect(...) # OK
88-
ctx.Schema("name") # OK
80+
ctx = dj.new(host="h", user="u", password="p") # OK
81+
ctx.config.safemode = False # OK
82+
ctx.Schema("name") # OK
8983
```
9084

9185
## Behavior Summary
9286

9387
| Operation | `thread_safe=False` | `thread_safe=True` |
9488
|-----------|--------------------|--------------------|
95-
| `dj.config` read | Works | Works |
96-
| `dj.config` write | Works | `ThreadSafetyError` |
89+
| `dj.config.thread_safe` | Works | Works |
90+
| `dj.config.*` (other) | Works | `ThreadSafetyError` |
9791
| `dj.conn()` | Works | `ThreadSafetyError` |
9892
| `dj.Schema()` | Works | `ThreadSafetyError` |
9993
| `dj.new()` | Works | Works |
100-
| `ctx.config` read/write | Works | Works |
101-
| `ctx.connect()` | Works | Works |
94+
| `ctx.config.*` | Works | Works |
95+
| `ctx.connection` | Works | Works |
10296
| `ctx.Schema()` | Works | Works |
10397

104-
## Context Lifecycle
98+
## Usage Example
10599

106100
```python
107-
# Create context
108-
ctx = dj.new()
109-
110-
# Configure
111-
ctx.config.database.host = "localhost"
112-
ctx.config.safemode = False
113-
ctx.config.stores = {...}
101+
import datajoint as dj
114102

115-
# Connect
116-
ctx.connect(
117-
host="localhost", # Or use ctx.config.database.host
103+
# Create isolated context
104+
ctx = dj.new(
105+
host="localhost",
118106
user="user",
119107
password="password",
120108
)
121109

122-
# Use
110+
# Configure
111+
ctx.config.safemode = False
112+
ctx.config.stores = {"raw": {"protocol": "file", "location": "/data"}}
113+
114+
# Create schema
123115
schema = ctx.Schema("my_schema")
124116

125117
@schema
126-
class Mouse(ctx.Manual):
118+
class Mouse(dj.Manual):
127119
definition = """
128120
mouse_id: int
129121
"""
130122

131-
Mouse().insert1({"mouse_id": 1})
132-
133-
# Cleanup (optional - closes connection)
134-
ctx.close()
135-
```
136-
137-
## Legacy Compatibility
138-
139-
The singleton `dj` context works exactly as before:
140-
141-
```python
142-
# These are equivalent:
143-
dj.conn() # Singleton connection
144-
dj.config # Singleton config
145-
dj.Schema("name") # Uses singleton connection
123+
@schema
124+
class Session(dj.Manual):
125+
definition = """
126+
-> Mouse
127+
session_date: date
128+
"""
146129

147-
# Internally, dj module delegates to singleton context
130+
# Use tables
131+
Mouse().insert1({"mouse_id": 1})
132+
Mouse().delete() # Uses ctx.config.safemode
148133
```
149134

150135
## Implementation
@@ -153,68 +138,46 @@ dj.Schema("name") # Uses singleton connection
153138

154139
```python
155140
class Context:
156-
def __init__(self, config: Config):
157-
self.config = config
158-
self._connection = None
159-
160-
def connect(self, host, user, password, ...):
161-
self._connection = Connection(...)
162-
self._connection.config = self.config
163-
164-
def conn(self):
165-
return self._connection
166-
167-
def Schema(self, name, ...):
168-
return Schema(name, connection=self._connection, ...)
169-
170-
# Table base classes that reference this context
171-
@property
172-
def Manual(self): ...
173-
@property
174-
def Lookup(self): ...
175-
# etc.
141+
def __init__(self, host, user, password, port=3306, ...):
142+
self.config = copy(dj.config) # Independent config copy
143+
self.connection = Connection(host, user, password, port, ...)
144+
self.connection._config = self.config # Link config to connection
145+
146+
def Schema(self, name, **kwargs):
147+
return Schema(name, connection=self.connection, **kwargs)
176148
```
177149

178150
### 2. Add dj.new()
179151

180152
```python
181-
def new() -> Context:
153+
def new(host, user, password, **kwargs) -> Context:
182154
"""Create a new isolated context with its own config and connection."""
183-
config_copy = copy(config) # Copy current global config
184-
return Context(config_copy)
155+
return Context(host, user, password, **kwargs)
185156
```
186157

187-
### 3. Make dj module act as singleton context
158+
### 3. Add thread_safe guards
159+
160+
In `dj.config`:
161+
- Allow read/write of `thread_safe` always
162+
- When `thread_safe=True`, block all other attribute access
188163

189164
```python
190-
# In datajoint/__init__.py
191-
_singleton_context = Context(config)
192-
193-
def conn(...):
194-
if config.thread_safe:
195-
raise ThreadSafetyError(...)
196-
return _singleton_context.conn(...)
197-
198-
def Schema(...):
199-
if config.thread_safe:
200-
raise ThreadSafetyError(...)
201-
return _singleton_context.Schema(...)
165+
def __getattr__(self, name):
166+
if name == "thread_safe":
167+
return self._thread_safe
168+
if self._thread_safe:
169+
raise ThreadSafetyError("Global config is inaccessible in thread-safe mode.")
170+
# ... normal access
202171
```
203172

204-
### 4. Add thread_safe guards
205-
206-
- `dj.conn()`: Raise `ThreadSafetyError` when `thread_safe=True`
207-
- `dj.Schema()`: Raise `ThreadSafetyError` when `thread_safe=True`
208-
- `dj.config` writes: Raise `ThreadSafetyError` when `thread_safe=True`
209-
210-
### 5. Refactor internal code
173+
### 4. Refactor internal code
211174

212-
All internal code uses `self.connection.config` instead of global `config`:
213-
- Tables access config via `self.connection.config`
214-
- Connection has reference to its context's config
175+
All internal code uses `self.connection._config` instead of global `config`:
176+
- Tables access config via `self.connection._config`
177+
- This works uniformly for both singleton and isolated contexts
215178

216179
## Error Messages
217180

218-
- `dj.conn()`: `"dj.conn() is disabled in thread-safe mode. Use ctx = dj.new() to create an isolated context."`
219-
- `dj.Schema()`: `"dj.Schema() is disabled in thread-safe mode. Use ctx = dj.new() to create an isolated context."`
220-
- `dj.config` write: `"Global config is read-only in thread-safe mode. Use ctx = dj.new() for isolated config."`
181+
- `dj.config.*`: `"Global config is inaccessible in thread-safe mode. Use ctx = dj.new(...) for isolated config."`
182+
- `dj.conn()`: `"dj.conn() is disabled in thread-safe mode. Use ctx = dj.new(...) to create an isolated context."`
183+
- `dj.Schema()`: `"dj.Schema() is disabled in thread-safe mode. Use ctx = dj.new(...) to create an isolated context."`

0 commit comments

Comments
 (0)