[drzraf]

Reroll of #304 (see initial discussion there) based on v3 branch.
See #295, #296 and #300

[AidasK]

are we still using states? I suggest to remove this old code and rely only on promises. Some breaking changes are welcome.

[drzraf]

Yes.

1. The the regular, non-async, readFileFn is untouched and still depends on state
2. isComplete depends on chunk.readState === 1 so we must still keep updating this value.

[AidasK]

We can remove regular readFileFn anyway, it's too complicated and same thing can be achieved via promises. Isn't it?

[bertrandg]

Hi, 👋

Thanks for your great work,
I've just discovered this v3 branch/issue #304 and i'm following it with lot of interest (will need to do file E2EE).

What's the current state of this v3 branch, is it currently "unfinished/break" or "working/need more tests" ?
What could I do to help you on it?
Do you plan to publish an npm "v3.0.0-beta" version soon?

[drzraf]

It's definitely usable (and developing a E2EE project already relying on it, also not -production ready).
As you know it's not only about async but about streams. What's already in, works, but other details must still be polished.

ES6 + class-split gave us the base. But subsequent code cleaning and improvements are needed:

• The (sensitive) part about concurrency could (and should) be made more readable by now.
• async initFileFn must be fixed (support async function for initFileFn #296)
• hooks should be discussed (async Init file function: streams and fileAdded events #319)
• Handling stream needs more polishing regarding file.size, predictable chunks number and progressbar (Pre-processing chunks and buffers #42 (comment))

[bertrandg]

Awesome, I will experiment with it and will tell you if I have some feedbacks 👍

[bertrandg]

Just to let you know, I've coded an e2ee POC with v3 and it works well 👍

Here is how i did with latest v3 commit:

function readFileChunk(file, startByte, endByte) {
    return new Promise(resolve => {
        const reader = new FileReader();
        reader.onload = () => {
            const bytes = new Uint8Array(reader.result);
            resolve(bytes);
        };
        const blob = file.slice(startByte, endByte);
        reader.readAsArrayBuffer(blob);
    });
}
  
async function encryptFileChunk(plaintextbytes, iv, key) {
    let cypherchunkbytes = await window.crypto.subtle.encrypt({name: 'AES-GCM', iv}, key, plaintextbytes);
  
    if(cypherchunkbytes) {
        cypherchunkbytes = new Uint8Array(cypherchunkbytes);
        return cypherchunkbytes;
    }
}
  
function init() {
    const flow = new Flow({
        testChunks: false,
        target: '/upload',
        chunkSize: window.chunkSize,
        allowDuplicateUploads: true,
        forceChunkSize: false,
        simultaneousUploads: 4,
        uploadMethod: 'POST',
        fileParameterName: 'file',
        asyncReadFileFn: async function(flowObj, startByte, endByte, fileType, chunk) {
            const plaintextbytes = await readFileChunk(flowObj.file, startByte, endByte);
            const cypherbytes = await encryptFileChunk(plaintextbytes, window.ivbytes, window.key);
    
            // Add 16 bytes from initialization vector
            chunk.chunkSize = chunk.chunkSize + 16; 
    
            const blob = new Blob([cypherbytes], {type: 'application/octet-stream'});
            return blob;
        }
    });
  
    flow.on('fileAdded', function(file) {
        // Add 16 bytes from each chunk initialization vector to total encrypted file size
        file.size += file.chunks.length * 16;
    });
}

[drzraf]

Very good. In my case I had the need for an async initFileFn because of openpgp encryption key initialization and full stream based encryption.
For that purpose: #329 you may want to test it too in case you want to avoid keeping the whole regular file in memory ;)

[AidasK]

@bertrandg Can you add this to our samples folder? It would be great to have it for others to see. It's enough just to make one samples/encryption.md file with some explanations

[bertrandg]

@drzraf I don't think I got the whole regular file in memory there..
I read/load only file chunk (const blob = file.slice(startByte, endByte); reader.readAsArrayBuffer(blob);) in asyncReadFileFn on the fly called before each chunk POST request.
Is the lib loading full file in memory?

@AidasK Yes sure, happy to help! :)