Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add interface to allow "more direct" access to the filecache without direct db #44044

Closed
wants to merge 4 commits into from

Conversation

icewind1991
Copy link
Member

@icewind1991 icewind1991 commented Mar 6, 2024

Adds and interface for the filecache that is lower level than the "normal" filesystem apis to allow for some more optimized retrieval of filecache data without requiring direct database access.

This will be needed when we start restricting direct database access to the filecache for sharding.

Pr comes with a few cases of replacing direct-db with the new interface.

extracted from #43576

apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
@icewind1991 icewind1991 force-pushed the remove-filecache-joins branch 3 times, most recently from 17ea359 to 39b1c7c Compare March 6, 2024 17:48
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
apps/files_sharing/lib/Updater.php Fixed Show fixed Hide fixed
@icewind1991 icewind1991 force-pushed the remove-filecache-joins branch 2 times, most recently from a14aa3a to 224966c Compare March 7, 2024 11:46
@icewind1991 icewind1991 force-pushed the remove-filecache-joins branch 2 times, most recently from 19ff9e6 to dc9729e Compare March 7, 2024 14:34
@icewind1991 icewind1991 force-pushed the remove-filecache-joins branch 3 times, most recently from f6dc6a2 to 3dc3ee4 Compare March 21, 2024 18:49
@icewind1991 icewind1991 changed the title Remove filecache joins Add interface to allow "more direct" access to the filecache without direct db Mar 21, 2024
@icewind1991 icewind1991 added the 3. to review Waiting for reviews label Mar 21, 2024
@icewind1991 icewind1991 added this to the Nextcloud 29 milestone Mar 21, 2024
@icewind1991 icewind1991 requested review from a team, Altahrim, artonge and Fenn-CS and removed request for a team March 21, 2024 18:53
@Altahrim Altahrim mentioned this pull request Mar 21, 2024
@icewind1991 icewind1991 marked this pull request as ready for review March 21, 2024 21:12
lib/private/Files/Cache/CacheAccess.php Outdated Show resolved Hide resolved
lib/private/Files/Cache/CacheAccess.php Outdated Show resolved Hide resolved
@Altahrim Altahrim mentioned this pull request Mar 25, 2024
$qb->andWhere($qb->expr()->eq('f.storage', $qb->createNamedParameter($node->getMountPoint()->getNumericStorageId(), IQueryBuilder::PARAM_INT)));
if ($shallow) {
$qb->andWhere($qb->expr()->eq('f.parent', $qb->createNamedParameter($node->getId(), IQueryBuilder::PARAM_INT)));
} else {
$qb->andWhere($qb->expr()->like('f.path', $qb->createNamedParameter($this->dbConnection->escapeLikeParameter($node->getInternalPath()) . '/%')));
}
$qb->andWhere($qb->expr()->eq('f.parent', $qb->createNamedParameter($node->getId())));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this change about?

Comment on lines +83 to +85
$shareManager->getSharesBy($user->getUID(), IShare::TYPE_USER),
$shareManager->getSharesBy($user->getUID(), IShare::TYPE_GROUP),
$shareManager->getSharesBy($user->getUID(), IShare::TYPE_ROOM),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did the $src parameter go?

@@ -0,0 +1,100 @@
<?php

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strict typing missing

->where($builder->expr()->eq('m.mount_point', $mountPoint))
->andWhere($builder->expr()->in('m.user_id', $builder->createNamedParameter($userIds, IQueryBuilder::PARAM_STR_ARRAY)));

$result = $query->execute();

$results = [];
while ($row = $result->fetch()) {
$results[$row['user_id']] = $row['size'];
$results[$row['user_id']] = $this->getSizeForHomeStorage($row['storage_id']);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means one query per row instead of getting all sizes in the first query, no?

@@ -0,0 +1,76 @@
<?php

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

strict typing

@icewind1991
Copy link
Member Author

Let me split out the interface and changes to remove filecache joins to simplify reviewing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3. to review Waiting for reviews
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants