-
Notifications
You must be signed in to change notification settings - Fork 9
feat: detect CAR files in add command and warn user #275
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| import { createWriteStream } from 'node:fs' | ||
| import { unlink } from 'node:fs/promises' | ||
| import { join } from 'node:path' | ||
| import { CarWriter } from '@ipld/car' | ||
| import { describe, expect, it } from 'vitest' | ||
| import { isCarFile } from '../../utils/car-detection.js' | ||
| import { CID } from 'multiformats/cid' | ||
| import * as raw from 'multiformats/codecs/raw' | ||
| import { sha256 } from 'multiformats/hashes/sha2' | ||
|
|
||
| describe('isCarFile', () => { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this filename should be updated as well, likely |
||
| const tempDir = process.cwd() | ||
| const validCarPath = join(tempDir, 'valid.car') | ||
| const invalidCarPath = join(tempDir, 'invalid.car') | ||
| const textFilePath = join(tempDir, 'text.txt') | ||
|
Comment on lines
+11
to
+15
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The tests here should be pretty simple and the same for each scenario. This file could be simplified a lot by doing some table-driven/data-driven test scenarios in an array of tuples, with a single it block: const tests = [
['valid.car', 'returns true for valid car files', true],
['invalid.car', 'returns false for invalid car files', false],
['text.txt', 'returns false for regular files', false]
]
describe('isCarFile', () => {
beforeAll(() => { /* set up all files */ })
afterAll(() => { /* cleanup */ })
for([fileName, msg, result] of tests) {
it(msg, () => {
// read file, call isCar, assert result
})
}
}) |
||
|
|
||
| it('should return true for a valid CAR file', async () => { | ||
| // Create a valid CAR file | ||
| const { out, writer } = await CarWriter.create([ | ||
| CID.create(1, raw.code, await sha256.digest(new Uint8Array([1, 2, 3]))) | ||
| ]) | ||
| const { Readable } = await import('node:stream') | ||
| const stream = createWriteStream(validCarPath) | ||
| await new Promise<void>((resolve, reject) => { | ||
| Readable.from(out).pipe(stream) | ||
| stream.on('error', reject) | ||
| stream.on('finish', () => resolve()) | ||
| writer.close() | ||
| }) | ||
|
|
||
| expect(await isCarFile(validCarPath)).toBe(true) | ||
| }) | ||
|
|
||
| it('should return false for a text file', async () => { | ||
| const stream = createWriteStream(textFilePath) | ||
| stream.write('This is just a text file') | ||
| stream.end() | ||
| await new Promise<void>((resolve) => stream.on('finish', () => resolve())) | ||
|
Comment on lines
+35
to
+38
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the two instances of non-CAR could just be a plain |
||
|
|
||
| expect(await isCarFile(textFilePath)).toBe(false) | ||
| }) | ||
|
|
||
| it('should return false for a random binary file', async () => { | ||
| const stream = createWriteStream(invalidCarPath) | ||
| stream.write(Buffer.from([0, 1, 2, 3, 4, 5])) | ||
| stream.end() | ||
| await new Promise<void>((resolve) => stream.on('finish', () => resolve())) | ||
|
|
||
| expect(await isCarFile(invalidCarPath)).toBe(false) | ||
| }) | ||
|
|
||
| // Cleanup | ||
| it('cleanup', async () => { | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This isn't a test so don't make it one, you have @SgtPooki might have more creative ideas of how to structure this though. |
||
| try { await unlink(validCarPath) } catch { } | ||
| try { await unlink(textFilePath) } catch { } | ||
| try { await unlink(invalidCarPath) } catch { } | ||
| }) | ||
| }) | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,20 @@ | ||||||
| import { createReadStream } from 'node:fs' | ||||||
| import { asyncIterableReader, createDecoder } from '@ipld/car/decoder' | ||||||
|
|
||||||
| export async function isCarFile(filePath: string): Promise<boolean> { | ||||||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. file name should match exported function.. |
||||||
| let stream: ReturnType<typeof createReadStream> | undefined | ||||||
|
|
||||||
| try { | ||||||
| stream = createReadStream(filePath) | ||||||
| const reader = asyncIterableReader(stream) | ||||||
| const decoder = createDecoder(reader) | ||||||
| const header = await decoder.header() | ||||||
| return !!(header && header.version === 1 && header.roots) | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. version can also be
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| } catch (error) { | ||||||
| return false | ||||||
| } finally { | ||||||
| if (stream) { | ||||||
| stream.destroy() | ||||||
| } | ||||||
| } | ||||||
| } | ||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you run this branch and do an
addfor a valid CAR and show us what the output looks like? then also do it and| lessthe output and show us that output too? I'm not convinced about the multi-line here, but also want to know that alog.warnactually does something useful.One for @SgtPooki to consider too, is this the right way we want to output a warning in here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For this scenario, I think we should display a
spinnerSectiontelling the users to "usefilecoin-pin importor provide paths to a non-CAR file or a directory" instead of usinglog.warn.log.*with the pino logger will only be visible if someone explicitly enables pino log levels.In general, inside the CLI "UI" code, i.e.
/src/commands/*we should display warnings explicitly via clack, similar topayments setup(non-auto) and potentially prompt them to confirm understanding.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should do this check from inside
runAdditself. There is already anawait validatePathinside there that determines if it's a directory or a file, we could type that as "contentType" instead of a simpleisDirectoryboolean, and pivot display messaging there.then we can move on to the future i'm imagining if Rod is in agreement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fine by me