Skip to content

Conversation

@benzekrimaha
Copy link
Contributor

Issue: CLDSRV-826

@bert-e
Copy link
Contributor

bert-e commented Jan 14, 2026

Hello benzekrimaha,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request TBA
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e
Copy link
Contributor

bert-e commented Jan 14, 2026

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-826 branch 2 times, most recently from 5baf746 to ea9cb65 Compare January 14, 2026 17:34
@benzekrimaha benzekrimaha marked this pull request as ready for review January 14, 2026 17:34
@benzekrimaha benzekrimaha requested review from a team, SylvainSenechal, delthas, francoisferrand and maeldonn and removed request for delthas and maeldonn January 14, 2026 17:35
@codecov
Copy link

codecov bot commented Jan 14, 2026

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
8070 2 8068 0
View the full list of 2 ❄️ flaky test(s)
"after each" hook for "should batch delete 1000 objects quietly"::Multi-Object Versioning Delete Success With default signature "after each" hook for "should batch delete 1000 objects quietly"

Flake rate in main: 100.00% (Passed 0 times, Failed 9 times)

Stack Traces | 0.099s run time
The bucket you tried to delete is not empty.
"before each" hook for "should batch delete 1000 objects quietly"::Multi-Object Versioning Delete Success With default signature "before each" hook for "should batch delete 1000 objects quietly"

Flake rate in main: 100.00% (Passed 0 times, Failed 9 times)

Stack Traces | 0.024s run time
socket hang up

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

return gcpClient.send(command)
.then(data => callback(null, data))
.catch(err => {
if (err && err.$metadata && err.$metadata.httpStatusCode &&
Copy link
Contributor

@SylvainSenechal SylvainSenechal Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to check if err exists here. I think err.$metadata.httpstatus code will always be defined.
Probably same for statusCode, gotta call the api once to verify but I feel like we shouldn't have to check if its defined or not

before(done => {
config = getRealAwsConfig(credentialOne);
gcpClient = new GCP(config);
gcpClient.listObjects = (params, callback) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to understand : this was available in Arsenal GcpService but you just removed it and now you are redefining it in all these files, why not keeping it in Arsenal ? I think the GcpService already has several functions that are used for tests only 🤔
Or is it because you want to reassign the statusCode ?

Copy link
Contributor

@SylvainSenechal SylvainSenechal Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Upon looking : consider updating the gcpClientRetry function : check err.$metadata.statuscode == 429 on top of the existing check err.statusCode == 429.
This way, can keep listObjects in Arsenal. (unless there are other reason I didn't see)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even if we added the $metadata.httpStatusCode check inside gcpClientRetry, we still need bespoke error munging in these raw-node tests. Rather than keeping a half-working helper in Arsenal plus a bunch of local overrides, I’d rather keep the shared GcpService minimal and let the tests own whatever extra plumbing they require.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • in tests, we should not need a generic wrapper to remap errors : each test is a single scenario, we can update that test to check the excepted result (both error or success)
  • even if we really absolutely need a helper, just introduce a regular function but do not modify the gcpClient object
  • can't we migrate each test instead of keeping this yet another AWS sdk v2 lookalike api?

Copy link
Contributor

@SylvainSenechal SylvainSenechal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to use real arsenal version in package json
Just one comment to address about the gcpClient.listObjects, maybe can consider a utility function at least, or even better keep it in Arsenal

@SylvainSenechal SylvainSenechal dismissed their stale review January 15, 2026 10:48

some more comments to address

@benzekrimaha benzekrimaha changed the title Improvement/cldsrv 826 Cloudserver adaptation post arsenal refacto Jan 19, 2026
@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-826 branch from b7e25d5 to bd529bb Compare January 19, 2026 16:38
Copy link
Contributor

@francoisferrand francoisferrand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in addition to the comments overall, there is something weird in these test: we use GCP client from arsenal, yet at the same time there is a re-implementation of GCP without the sdk (c.f. makeGcpRequest, retryGcpRequest...)

this should probably be changed, to do things consistently: either with GCP client (to maximize code reuse), or with a custom implementation, or directly with the GCP sdk (if we want to ensure the test code is independent form the production code, and failure would thus only happen due to change in production code)

Comment on lines 19 to 47
gcpClient.listObjects = (params, callback) => {
const command = new ListObjectsCommand(params);
return gcpClient.send(command)
.then(data => callback(null, data))
.catch(err => {
if ( err.statusCode === undefined) {
// eslint-disable-next-line no-param-reassign
err.statusCode = err.$metadata.httpStatusCode;
}
return callback(err);
});
};

gcpClient.getBucket = (params, callback) =>
gcpClient.headBucket(params, (err, res) => {
if (err) {
if (err.statusCode === undefined) {
// eslint-disable-next-line no-param-reassign
err.statusCode = err.$metadata.httpStatusCode;
}
if (err.$metadata && err.$metadata.httpStatusCode === 404) {
// eslint-disable-next-line no-param-reassign
err.name = 'NoSuchBucket';
}
return callback(err);
}
return callback(null, res);
});

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why keep these "legacy" functions, and not migrate the actual tests to the new SDK api?
since this PR is focused, it does not seem like a large change...

config = getRealAwsConfig(credentialOne);
gcpClient = new GCP(config);
gcpClient.listObjects = (params, callback) => {
const command = new ListObjectsCommand(params);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above, should migrate each test? (and avoid duplicating this piece of code)

@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-826 branch 2 times, most recently from c3be2d7 to 9c34aa1 Compare January 27, 2026 15:49
process.stdout.write(`err in creating bucket ${err}\n`);
}
return done(err);
const command = new CreateBucketCommand({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there used to be a gcpRequestRetry() : no problem to stop using it, but don't we need to setup retry somehow? or is it useless and handled in the client now?

Comment on lines 78 to 87
before(done => {
gcpRequestRetry({
method: 'PUT',
bucket: bucketName,
authCredentials: config.credentials,
}, 0, err => {
if (err) {
process.stdout.write(`err in creating bucket ${err}\n`);
}
return done(err);
const command = new CreateBucketCommand({
Bucket: bucketName,
});
gcpClient.send(command)
.then(() => done())
.catch(err => {
process.stdout.write(`err in creating bucket ${err}\n`);
return done(err);
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    before(async () => {
        const command = new CreateBucketCommand({Collapse commentComment on line 
            Bucket: bucketName,
        });
        await gcpClient.send(command);
    });

Comment on lines 90 to 100
after(done => {
gcpRequestRetry({
method: 'DELETE',
bucket: bucketName,
authCredentials: config.credentials,
}, 0, err => {
if (err) {
process.stdout.write(`err in deleting bucket ${err}\n`);
}
return done(err);
const command = new DeleteBucketCommand({
Bucket: bucketName,
});
gcpClient.send(command)
.then(() => done())
.catch(err => {
process.stdout.write(`err in deleting bucket ${err}\n`);
return done(err);
});
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    after(async () => {
        const command = new DeleteBucketCommand({
            Bucket: bucketName,
        });
        await gcpClient.send(command)
    });

const config = getRealAwsConfig(credentialOne);
const gcpClient = new GCP(config);

beforeEach(function beforeFn(done) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make function async and use await when sending command (same for afterEach)

Comment on lines 52 to 77
it('should verify bucket versioning is enabled', function testFn(done) {
return async.waterfall([
next => makeGcpRequest({
method: 'PUT',
bucket: this.test.bucketName,
authCredentials: config.credentials,
queryObj: { versioning: '' },
requestBody: xmlEnable,
}, err => {
if (err) {
process.stdout.write(`err in setting versioning ${err.code}`);
}
return next(err);
}),
// Enable versioning using the official SDK client
next => {
gcpClient.getBucketVersioning({
const command = new PutBucketVersioningCommand({
Bucket: this.test.bucketName,
}, (err, res) => {
assert.equal(err, null,
`Expected success, but got err ${err}`);
assert.deepStrictEqual(res.Status, verEnabledObj);
return next();
VersioningConfiguration: { Status: 'Enabled' },
});
return gcpClient.send(command)
.then(() => next())
.catch(err => next(err));
},
// Verify using GetBucketVersioningCommand
next => {
const command = new GetBucketVersioningCommand({
Bucket: this.test.bucketName,
});
return gcpClient.send(command)
.then(res => {
assert.deepStrictEqual(res.Status, verEnabledObj);
return next();
})
.catch(err => next(err));
},
], err => done(err));
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

almost every line is changed, best use async and shorten the test:

    it('should verify bucket versioning is enabled', async () => {
           await gcpClient.send(new PutBucketVersioningCommand({
                Bucket: this.test.bucketName,
                VersioningConfiguration: { Status: 'Enabled' },
           }));

           const res = gcpClient.send(new GetBucketVersioningCommand({
                Bucket: this.test.bucketName,
           }));
           assert.deepStrictEqual(res.Status, verEnabledObj);
    });

},
err => {
if (err && (err.name === 'SlowDown'
|| err.$metadata?.httpStatusCode === 429)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bad indent, this is not at the same indent level as 'err' (nested expression)

Suggested change
|| err.$metadata?.httpStatusCode === 429)
|| err.$metadata?.httpStatusCode === 429)

or better yet just keep it on previous line, the line does not seem that long?

Comment on lines 29 to 50
function createBucket(attempt) {
const cmd = new CreateBucketCommand({ Bucket: bucketName });
gcpClient.send(cmd)
.then(() => {
bucketCreated = true;
return done();
})
.catch(err => {
if ((err.name === 'SlowDown'
|| err.$metadata?.httpStatusCode === 429)
&& attempt < maxAttempts - 1) {
const delay = Math.pow(2, attempt) * 1000;
process.stdout.write(
`SlowDown creating bucket, retrying in ${delay}ms (attempt ${attempt + 1})\n`);
return setTimeout(() => createBucket(attempt + 1), delay);
}
process.stdout.write(`err in creating bucket ${err}`);
return done(err);
});
}

createBucket(0);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't this be written much more cleanly with async.retry or even just modifying the gcpRequestRetry to take both a command and client and repeatedly send the command if needed... (that function would not really be specific to gcp, but provide a good separation between retry logic and the test)


const genUniqID = () => uuidv4().replace(/-/g, '');

function gcpRequestRetry(params, retry, callback) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • should still keep the retry logic isolated in some helper, instead of re-implementing directly in the tests
  • helper may need to support both async & callbacks (depending on where it is used)
async function gcpRetry(gcpClient, cmd, retry, cb) {
   if (cb) {
      return callbackify(() => gcpRetry(gcpClient, cmd, retry))(cb);
   }
   [...]
   return await gcpCient.send(cmd)
}

return callback(null, res);
});
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we really need helper (listObjects, ...) they should be added here

package.json Outdated
"@hapi/joi": "^17.1.1",
"@smithy/node-http-handler": "^3.0.0",
"arsenal": "git+https://github.com/scality/Arsenal#8.3.0-preview.1",
"arsenal": "git+https://github.com/scality/Arsenal#aed443f02efebeb3f5c2e74786baadb5c19eaded",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be updated with arsenal 8.3.0 now that it is released

@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-826 branch 2 times, most recently from 2e581a8 to cfe0d6a Compare January 29, 2026 15:32
@scality scality deleted a comment from bert-e Jan 29, 2026
@benzekrimaha benzekrimaha force-pushed the improvement/CLDSRV-826 branch from cfe0d6a to 884e1da Compare January 29, 2026 15:35
@bert-e
Copy link
Contributor

bert-e commented Jan 29, 2026

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • 2 peers

This commit removes unused functions and updates object tests accordingly.
These changes help reduce the dependency on gcpService only used functions in
cloudserver tests, and also enables us to remove wrappers around GCP SDK calls,
making the tests more straightforward and easier to maintain.
Issue: CLDSRV-826
Copy link
Contributor

@DarkIsDude DarkIsDude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a lot to say, code is clean with some async/await, congrats 🙏

async.mapLimit(
createdObjects,
10,
(object, moveOn) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: async module supports async functions, so the same can be written with async/await:

    async.mapLimit(
        createdObjects,
        10,
        async (object) => await gcClient.send(new PutObjectCommand(...)),
        callback,
    );

Comment on lines 28 to 29
const cmd = typeof makeCommand === 'function' ?
makeCommand() : makeCommand;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to support a function? There does not seem to be any case where the command is dynamically built.
can't we always receive the actual command to send, and just use it?

(and if we really needed to use a function, no point supporting both a function and a command)

before(async () => {
await gcpRetry(
gcpClient,
() => new CreateBucketCommand({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the point of passing a function instead of just the command object?
can't we keep things simple?


describe('without existing bucket', () => {
beforeEach(function beforeFn(done) {
beforeEach(function beforeFn() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not aync and no callback : will never terminate?

Suggested change
beforeEach(function beforeFn() {
beforeEach(async function beforeFn() {

Comment on lines 57 to 60
} catch (err) {
process.stdout
.write(`err deleting bucket: ${err.code}\n`);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the log is not needed (handled by Jest already), and more importantly this catch actually "eats" the error : so failure will not cause the test to fail (as it did before)

Suggested change
} catch (err) {
process.stdout
.write(`err deleting bucket: ${err.code}\n`);
}
await gcpClient.send(cmd);

Bucket: bucketName,
Key: this.currentTest.key,
});
gcpClient.send(cmd)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await

Bucket: bucketName,
Key: this.currentTest.key,
});
gcpClient.send(cmd)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await

Comment on lines 63 to 73
gcpRetry(
gcpClient,
() => new DeleteBucketCommand({ Bucket: bucketName }),
null,
err => {
if (err) {
process.stdout.write(`err in deleting bucket ${err}`);
}
return done(err);
},
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
gcpRetry(
gcpClient,
() => new DeleteBucketCommand({ Bucket: bucketName }),
null,
err => {
if (err) {
process.stdout.write(`err in deleting bucket ${err}`);
}
return done(err);
},
);
after(async () => {
await gcpRetry(
gcpClient,
new DeleteBucketCommand({ Bucket: bucketName }),
);

Comment on lines 39 to 44
(bucket, next) => gcpRetry(
gcpClient,
() => new CreateBucketCommand({ Bucket: bucket.Name }),
null,
next,
),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
(bucket, next) => gcpRetry(
gcpClient,
() => new CreateBucketCommand({ Bucket: bucket.Name }),
null,
next,
),
async (bucket) => await gcpRetry(
gcpClient,
() => new CreateBucketCommand({ Bucket: bucket.Name }),
),

'</CreateBucketConfiguration>';
}

function listBucketObjects(gcpClient, params, cb) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we really need this helper?
it is really just sending the command, we can migrate the 2 places where it is called, like everywhere else...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this change was made based on this comment here : #6044 (comment) to avoid having to redefine the functions everytime. switching back to the commands on the tests level.

fix awaiting on before/after
fix awaiting on before/after
fix awaiting on before/after
fix awaiting on before/after
fix awaiting on before/after on completeMPU
fix awaiting on before/after on completeMPU
fix awaiting on before/after on initiateMPU
fix awaiting on before/after on initiateMPU
fix awaiting on before/after on initiateMPU
fix awaiting on before/after on initiateMPU
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants