Skip to content

fix(bitbucket): Bitbucket Cloud pagination not working beyond first page #502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix(bitbucket): Bitbucket Cloud pagination not working beyond first page #502

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 24 additions & 8 deletions packages/backend/src/bitbucket.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,21 +170,33 @@ const getPaginatedCloud = async <T>(
}
return results;
}


/**
* Parse the url into a path and query parameters to be used with the api client (openapi-fetch)
*/
function parseUrl(url: string, baseUrl: string): { path: string; query: Record<string, string>; } {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we are using this in all get callbacks passed to getPaginatedCloud to handle response.next, I wonder if we can just move this logic into getPaginatedCloud directly?

For example, the signature could be:

const getPaginatedCloud = async <T>(
    path: CloudGetRequestPath,
    get: (path: CloudGetRequestPath, query?: string) => Promise<CloudPaginatedResponse<T>>
): Promise<T[]>

And getPaginatedCloud handles deconstructing response.next into path and optionally query.

const fullUrl = new URL(url);
Copy link
Contributor

@brendan-kellam brendan-kellam Sep 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bardock I tested and found the following error:

[dev:backend ] 2025-09-10T18:08:08.875Z error: [bitbucket] Failed to get repos for workspace facebot: TypeError: Invalid URL [dev:backend ] 2025-09-10T18:08:08.876Z error: [connection-manager] Failed to compile repo data for connection 1 (starter-connection): TypeError: Invalid URL

Apparently, the baseUrl is not being used, and when adjusting it, the issue of duplicating /2.0 occurs.

RE @WendelBartzUbots @bardock - best guess is that url is /repositories/${workspace} (and not the full url) for the first call of the callback provided to getPaginatedCloud. I think the approach I outlined above should fix this (i.e., just deconstruct response.next and not path)

const path = fullUrl.pathname;
const query = Object.fromEntries(fullUrl.searchParams);
logger.debug(`Parsed url ${url} into path ${path} and query ${JSON.stringify(query)}`);
return { path, query };
}


async function cloudGetReposForWorkspace(client: BitbucketClient, workspaces: string[]): Promise<{validRepos: CloudRepository[], notFoundWorkspaces: string[]}> {
const results = await Promise.allSettled(workspaces.map(async (workspace) => {
try {
logger.debug(`Fetching all repos for workspace ${workspace}...`);

const path = `/repositories/${workspace}` as CloudGetRequestPath;
const { durationMs, data } = await measure(async () => {
const fetchFn = () => getPaginatedCloud<CloudRepository>(path, async (url) => {
const response = await client.apiClient.GET(url, {
const fetchFn = () => getPaginatedCloud<CloudRepository>(`/repositories/${workspace}` as CloudGetRequestPath, async (url) => {
const { path, query } = parseUrl(url, client.baseUrl);
const response = await client.apiClient.GET(path, {
params: {
path: {
workspace,
}
},
query: query,
}
});
const { data, error } = response;
Expand Down Expand Up @@ -238,11 +250,15 @@ async function cloudGetReposForProjects(client: BitbucketClient, projects: strin

logger.debug(`Fetching all repos for project ${project} for workspace ${workspace}...`);
try {
const path = `/repositories/${workspace}` as CloudGetRequestPath;
const repos = await getPaginatedCloud<CloudRepository>(path, async (url) => {
const response = await client.apiClient.GET(url, {
const repos = await getPaginatedCloud<CloudRepository>(`/repositories/${workspace}` as CloudGetRequestPath, async (url) => {
const { path, query } = parseUrl(url, client.baseUrl);
const response = await client.apiClient.GET(path, {
params: {
path: {
workspace,
},
query: {
...query,
q: `project.key="${project_name}"`
}
}
Expand Down
Loading