refactor: remove dependency on global Buffer in js

[olehmisar]

Summary

Removed Buffer global mentions in the whole codebase by replacing it with bufferAlloc, bufferFrom and similar utils. It still uses Buffer but now imported from the buffer npm package. The main files to review are foundation/src/buffer/utils.ts and playground/*.

Caveats

1. import('buffer').Buffer differs from nodejs Buffer. It does NOT support base64url encoding. Does not appear to support 'base64url' encoding like Node Buffer feross/buffer#309
2. writeBigUint64BE (with lower "i") is not supported writeBigUint64BE isn't defined, but writeBigUInt64BE is feross/buffer#318

How it is done

Since this is a large change from an external contributor(a security risk), here are the steps I took to implement it, so an internal employee can safely do the same in 5-10 minutes.

1. Almost the whole PR is the result of running the following script (commit 62eca82):

open

cd yarn-project

git grep -l "Buffer.from(" | xargs -r sed -i -e 's/Buffer.from(/bufferFrom(/g;1{/^#!/{n;s|^|import { bufferFrom } from "@aztec/foundation/buffer";\n|};/^#!/!s|^|import { bufferFrom } from "@aztec/foundation/buffer";\n|}'
git grep -l "Buffer.alloc(" | xargs -r sed -i -e 's/Buffer.alloc(/bufferAlloc(/g;1{/^#!/{n;s|^|import { bufferAlloc } from "@aztec/foundation/buffer";\n|};/^#!/!s|^|import { bufferAlloc } from "@aztec/foundation/buffer";\n|}'
git grep -l "Buffer.concat(" | xargs -r sed -i -e 's/Buffer.concat(/bufferConcat(/g;1{/^#!/{n;s|^|import { bufferConcat } from "@aztec/foundation/buffer";\n|};/^#!/!s|^|import { bufferConcat } from "@aztec/foundation/buffer";\n|}'
git grep -l "Buffer.isBuffer(" | xargs -r sed -i -e 's/Buffer.isBuffer(/isBuffer(/g;1{/^#!/{n;s|^|import { isBuffer } from "@aztec/foundation/buffer";\n|};/^#!/!s|^|import { isBuffer } from "@aztec/foundation/buffer";\n|}'
git grep -l "Buffer.compare" | xargs -r sed -i -e 's/\bBuffer\.compare\b/bufferCompare/g;1{/^#!/{n;s|^|import { bufferCompare } from "@aztec/foundation/buffer";\n|};/^#!/!s|^|import { bufferCompare } from "@aztec/foundation/buffer";\n|}'

node <<'NODE'
const { execSync } = require('child_process');
const fs = require('fs');
const path = require('path');

const base = path.join(process.cwd(), 'foundation');
const target = path.join(base, 'src/buffer/index.js');

for (const f of execSync('git ls-files', { cwd: base, encoding: 'utf8' }).trim().split('\n')) {
  const p = path.join(base, f);
  if (!fs.existsSync(p)) continue;
  let s = fs.readFileSync(p, 'utf8');
  if (!s.includes('from "@aztec/foundation/buffer";')) continue;

  const rel = path.posix.relative(
    path.posix.dirname(p.split(path.sep).join('/')),
    target.split(path.sep).join('/')
  );

  s = s.replaceAll(
    'from "@aztec/foundation/buffer";',
    `from "${rel.startsWith('.') ? rel : './' + rel}";`
  );

  fs.writeFileSync(p, s);
}
NODE

cd foundation/src/buffer
cat <<'EOF' > utils.ts
import { Buffer } from 'buffer';

export function bufferAlloc(size: number, fill?: number | string | Uint8Array): Buffer {
  return Buffer.alloc(size, fill);
}

// the overloads are too complex, so just re-assign
export const bufferFrom = Buffer.from;

export function bufferConcat(list: readonly Uint8Array[], totalLength?: number) {
  return Buffer.concat(list, totalLength);
}

export function isBuffer(obj: any) {
  return Buffer.isBuffer(obj);
}

export function bufferCompare(a: Uint8Array, b: Uint8Array) {
  return Buffer.compare(a, b);
}
EOF

printf "export * from './utils.js';\n" >> index.ts

cd ../../..
cd foundation && yarn add buffer
cd ..

yarn lint
yarn format

2. Then post fixes (commit e0db90a)

3. And then removed the polyfills from the playground (commit de0aff7)

TODO

• check if playground actually works. I was able to run it but couldn't send a tx because the local network won't start due to @trackSpan(function () { SyntaxError: Invalid or unexpected token when I tried to yarn start --local-network inside the yarn-project/aztec.
• an eslint rule to prevent reintroducing Buffer.xxx usage