Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions doc/api/sqlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -1075,8 +1075,10 @@ changes:
-->

* `sourceDb` {DatabaseSync} The database to backup. The source database must be open.

* `path` {string | Buffer | URL} The path where the backup will be created. If the file already exists,
the contents will be overwritten.

* `options` {Object} Optional configuration for the backup. The
following properties are supported:
* `source` {string} Name of the source database. This can be `'main'` (the default primary database) or any other
Expand All @@ -1087,6 +1089,10 @@ changes:
* `progress` {Function} An optional callback function that will be called after each backup step. The argument passed
to this callback is an {Object} with `remainingPages` and `totalPages` properties, describing the current progress
of the backup operation.
* `signal` {AbortSignal} An optional AbortSignal that can be used to abort the backup operation.\
If the signal is aborted, the backup operation will be cancelled and the Promise will reject with an `AbortError`.\
**Note:** Aborting is a best-effort mechanism; the backup may complete before the abort is processed due to race
conditions.
* Returns: {Promise} A promise that fulfills with the total number of backed-up pages upon completion, or rejects if an
error occurs.

Expand Down Expand Up @@ -1127,6 +1133,34 @@ const totalPagesTransferred = await backup(sourceDb, 'backup.db', {
console.log('Backup completed', totalPagesTransferred);
```

### Aborting a backup

The backup operation can be cancelled using an `AbortSignal`:

```cjs
const { backup, DatabaseSync } = require('node:sqlite');

(async () => {
const sourceDb = new DatabaseSync('source.db');

try {
await backup(sourceDb, 'backup.db', {
signal: AbortSignal.timeout(5000),
progress: ({ totalPages, remainingPages }) => {
console.log('Backup in progress', { totalPages, remainingPages });
},
});
console.log('Backup completed successfully');
} catch (error) {
if (error.name === 'AbortError') {
console.log('Backup was cancelled:', error.message);
} else {
console.error('Backup failed:', error.message);
}
}
})();
```

## `sqlite.constants`

<!-- YAML
Expand Down
28 changes: 27 additions & 1 deletion lib/sqlite.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,32 @@
'use strict';
const {
PromiseReject,
} = primordials;
const {
AbortError,
} = require('internal/errors');
const {
validateAbortSignal,
validateObject,
validateString,
} = require('internal/validators');
const { emitExperimentalWarning } = require('internal/util');

emitExperimentalWarning('SQLite');

module.exports = internalBinding('sqlite');
const binding = internalBinding('sqlite');

function backup(sourceDb, path, options = {}) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this function all in C++

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering why we would need to send it to the C++ layer if it is not valid. On the other hand, it might look cleaner to leave this file as it is and add everything in C++ instead.

validateObject(sourceDb, 'sourceDb');
validateString(path, 'options.headers.host');
validateAbortSignal(options.signal, 'options.signal');
if (options.signal?.aborted) {
return PromiseReject(new AbortError(undefined, { cause: options.signal.reason }));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One issue I can see with this, which does not need to be addressed now, is that this will mean we have two AbortError types, one defined in JavaScript another defined in C++. They'll both be named AbortError but won't be the same class so instanceof checks between the two won't work. Not a big deal but it might be good to leave a todo somewhere for someone later to reconcile the two into a single implementation.

Copy link
Contributor Author

@lluisemper lluisemper Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out, I wasn't sure if I understood this correctly when I was told in a previous comment! I'd like to follow up on this after landing the current PR. It definitely makes sense to reconcile the two AbortError implementations so instanceof works consistently from js.

Would you happen to have any pointers on the preferred approach for reusing the js native AbortError in C++?

I have put the comment here: https://github.com/nodejs/node/pull/59333/files#diff-dd810db4fe69364c3a67a274e0725f386040c0fd1dcfade7093f23c8514328aeR119-R120

}
return binding.backup(sourceDb, path, options);
}

module.exports = {
...binding,
backup,
};
4 changes: 4 additions & 0 deletions src/env_properties.h
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,10 @@
V(readable_string, "readable") \
V(read_bigints_string, "readBigInts") \
V(reason_string, "reason") \
V(cause_string, "cause") \
V(aborted_string, "aborted") \
V(refresh_string, "refresh") \
V(regexp_string, "regexp") \
V(remaining_pages_string, "remainingPages") \
V(rename_string, "rename") \
V(required_module_facade_url_string, \
Expand Down
165 changes: 155 additions & 10 deletions src/node_sqlite.cc
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include "threadpoolwork-inl.h"
#include "util-inl.h"

#include <atomic>
#include <cinttypes>

namespace node {
Expand Down Expand Up @@ -120,6 +121,55 @@ using v8::Value;
UNREACHABLE("Bad SQLite value"); \
} \
} while (0)
// TODO(@lluisemper) This is a copy of node::AbortError, use js native
// AbortError constructor to allow instanceof checks in JS.
class AbortError {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@himself65 I have tried to implement this class as you suggested, I hope I have not misunderstood 😂 I have check its js implementation from here: https://github.com/nodejs/node/blob/main/lib/internal/errors.js#L976-L988

public:
static MaybeLocal<Object> New(
Isolate* isolate,
std::string_view message = "The operation was aborted",
Local<Value> cause = Local<Value>()) {
Local<String> js_msg;
Local<Object> error_obj;
Local<Context> context = isolate->GetCurrentContext();
Environment* env = Environment::GetCurrent(isolate);

if (!String::NewFromUtf8(isolate,
message.data(),
NewStringType::kNormal,
static_cast<int>(message.size()))
.ToLocal(&js_msg) ||
!Exception::Error(js_msg)->ToObject(context).ToLocal(&error_obj)) {
return MaybeLocal<Object>();
}

Local<String> error_name;
if (!String::NewFromUtf8(isolate, "AbortError").ToLocal(&error_name)) {
return MaybeLocal<Object>();
}
if (error_obj->Set(context, env->name_string(), error_name).IsNothing()) {
return MaybeLocal<Object>();
}

Local<String> code_key;
Local<String> code_value;
if (!String::NewFromUtf8(isolate, "code").ToLocal(&code_key) ||
!String::NewFromUtf8(isolate, "ABORT_ERR").ToLocal(&code_value)) {
return MaybeLocal<Object>();
}
if (error_obj->Set(context, code_key, code_value).IsNothing()) {
return MaybeLocal<Object>();
}

if (!cause.IsEmpty() && !cause->IsUndefined()) {
if (error_obj->Set(context, env->cause_string(), cause).IsNothing()) {
return MaybeLocal<Object>();
}
}

return error_obj;
}
};

namespace {
Local<DictionaryTemplate> getLazyIterTemplate(Environment* env) {
Expand All @@ -134,11 +184,16 @@ Local<DictionaryTemplate> getLazyIterTemplate(Environment* env) {
} // namespace

inline MaybeLocal<Object> CreateSQLiteError(Isolate* isolate,
const char* message) {
std::string_view message) {
Local<String> js_msg;
Local<Object> e;
Environment* env = Environment::GetCurrent(isolate);
if (!String::NewFromUtf8(isolate, message).ToLocal(&js_msg) ||

if (!String::NewFromUtf8(isolate,
message.data(),
NewStringType::kNormal,
static_cast<int>(message.size()))
.ToLocal(&js_msg) ||
!Exception::Error(js_msg)
->ToObject(isolate->GetCurrentContext())
.ToLocal(&e) ||
Expand All @@ -148,6 +203,7 @@ inline MaybeLocal<Object> CreateSQLiteError(Isolate* isolate,
.IsNothing()) {
return MaybeLocal<Object>();
}

return e;
}

Expand Down Expand Up @@ -450,16 +506,21 @@ class BackupJob : public ThreadPoolWork {
std::string destination_name,
std::string dest_db,
int pages,
Local<Function> progressFunc)
Local<Function> progress_func,
Local<Object> abort_signal = Local<Object>())
: ThreadPoolWork(env, "node_sqlite3.BackupJob"),
env_(env),
source_(source),
pages_(pages),
source_db_(std::move(source_db)),
destination_name_(std::move(destination_name)),
dest_db_(std::move(dest_db)) {
dest_db_(std::move(dest_db)),
is_aborted_(false) {
resolver_.Reset(env->isolate(), resolver);
progressFunc_.Reset(env->isolate(), progressFunc);
progress_func_.Reset(env->isolate(), progress_func);
if (!abort_signal.IsEmpty()) {
abort_signal_.Reset(env->isolate(), abort_signal);
}
}

void ScheduleBackup() {
Expand Down Expand Up @@ -488,6 +549,10 @@ class BackupJob : public ThreadPoolWork {
}

void DoThreadPoolWork() override {
if (is_aborted_.load(std::memory_order_acquire)) {
backup_status_ = SQLITE_INTERRUPT;
return;
}
backup_status_ = sqlite3_backup_step(backup_, pages_);
}

Expand All @@ -496,6 +561,12 @@ class BackupJob : public ThreadPoolWork {
Local<Promise::Resolver> resolver =
Local<Promise::Resolver>::New(env()->isolate(), resolver_);

if (is_aborted_.load(std::memory_order_acquire) ||
backup_status_ == SQLITE_INTERRUPT) {
HandleAbortError(resolver);
return;
}

if (!(backup_status_ == SQLITE_OK || backup_status_ == SQLITE_DONE ||
backup_status_ == SQLITE_BUSY || backup_status_ == SQLITE_LOCKED)) {
HandleBackupError(resolver, backup_status_);
Expand All @@ -506,7 +577,7 @@ class BackupJob : public ThreadPoolWork {
int remaining_pages = sqlite3_backup_remaining(backup_);
if (remaining_pages != 0) {
Local<Function> fn =
Local<Function>::New(env()->isolate(), progressFunc_);
Local<Function>::New(env()->isolate(), progress_func_);
if (!fn.IsEmpty()) {
Local<Object> progress_info = Object::New(env()->isolate());
if (progress_info
Expand All @@ -531,6 +602,14 @@ class BackupJob : public ThreadPoolWork {
return;
}
}
if (CheckAbortSignal()) {
// TODO(@lluisemper): BackupJob does not implement proper async context
// tracking yet.
// Consider inheriting from AsyncWrap and using CallbackScope to
// propagate async context, similar to other ThreadPoolWork items.
HandleAbortError(resolver);
return;
}

// There's still work to do
this->ScheduleWork();
Expand Down Expand Up @@ -564,6 +643,10 @@ class BackupJob : public ThreadPoolWork {
sqlite3_close_v2(dest_);
dest_ = nullptr;
}

if (!abort_signal_.IsEmpty()) {
abort_signal_.Reset();
}
}

private:
Expand All @@ -589,19 +672,73 @@ class BackupJob : public ThreadPoolWork {
resolver->Reject(env()->context(), e).ToChecked();
}

inline MaybeLocal<Object> CreateAbortError(
Isolate* isolate,
std::string_view message = "The operation was aborted") {
Environment* env = Environment::GetCurrent(isolate);
HandleScope scope(isolate);
Local<Value> cause;

if (!abort_signal_.IsEmpty()) {
Local<Object> signal = abort_signal_.Get(isolate);
Local<String> reason_key = env->reason_string();

if (!signal->Get(isolate->GetCurrentContext(), reason_key)
.ToLocal(&cause)) {
cause = Local<Value>();
}
}

return AbortError::New(isolate, message, cause);
}

void HandleAbortError(Local<Promise::Resolver> resolver) {
Local<Object> e;
if (!CreateAbortError(env()->isolate()).ToLocal(&e)) {
Finalize();
return;
}

Finalize();
resolver->Reject(env()->context(), e).ToChecked();
}

bool CheckAbortSignal() {
if (abort_signal_.IsEmpty()) {
return false;
}

Isolate* isolate = env()->isolate();
HandleScope scope(isolate);
Local<Object> signal = abort_signal_.Get(isolate);

Local<Value> aborted_value;
if (signal->Get(env()->context(), env()->aborted_string())
.ToLocal(&aborted_value)) {
if (aborted_value->BooleanValue(isolate)) {
is_aborted_.store(true, std::memory_order_release);
return true;
}
}

return false;
}

Environment* env() const { return env_; }

Environment* env_;
DatabaseSync* source_;
Global<Promise::Resolver> resolver_;
Global<Function> progressFunc_;
Global<Function> progress_func_;
Global<Object> abort_signal_;
sqlite3* dest_ = nullptr;
sqlite3_backup* backup_ = nullptr;
int pages_;
int backup_status_ = SQLITE_OK;
std::string source_db_;
std::string destination_name_;
std::string dest_db_;
std::atomic<bool> is_aborted_;
};

UserDefinedFunction::UserDefinedFunction(Environment* env,
Expand Down Expand Up @@ -1597,7 +1734,8 @@ void Backup(const FunctionCallbackInfo<Value>& args) {
int rate = 100;
std::string source_db = "main";
std::string dest_db = "main";
Local<Function> progressFunc = Local<Function>();
Local<Function> progress_func = Local<Function>();
Local<Object> abort_signal = Local<Object>();

if (args.Length() > 2) {
if (!args[2]->IsObject()) {
Expand Down Expand Up @@ -1669,7 +1807,13 @@ void Backup(const FunctionCallbackInfo<Value>& args) {
"The \"options.progress\" argument must be a function.");
return;
}
progressFunc = progress_v.As<Function>();
progress_func = progress_v.As<Function>();
}

Local<Value> signal_v;
if (!options->Get(env->context(), env->signal_string())
.ToLocal(&signal_v)) {
return;
}
}

Expand All @@ -1686,7 +1830,8 @@ void Backup(const FunctionCallbackInfo<Value>& args) {
dest_path.value(),
std::move(dest_db),
rate,
progressFunc);
progress_func,
abort_signal);
db->AddBackup(job);
job->ScheduleBackup();
}
Expand Down
Loading
Loading