3 Commits (35ac2bb6a678106b470468d5f250b29dd6598e94)

Author	SHA1	Message	Date
cassiozareck	3198b4a642	[FEAT] add Forgero Git Service ... Signed-off-by: cassiozareck <cassiomilczareck@gmail.com> (cherry picked from commit a878adfe628cf6dc367a17c3715fcd3499aa02b6) Adding description and Forgejo SVG (cherry picked from commit 13738c03804d019f28550e46a4ebc37dbe3a5cfc) Undo reordering and tmpl redirection (cherry picked from commit 9ae51c46f42acecac834371857e638098ebf6d27) (cherry picked from commit 70fffdc61d06dd1d70b6a31496676a23d3d0c2fc) (cherry picked from commit c0ebfa9da3db3e60d7b403a1bf8b8a19c32c5dc7) (cherry picked from commit 9922c92787eccaba0021486ba0a3eb28583969e1) (cherry picked from commit 00c0effbc74aedc7a4167a69c8a410ef324d576b) (cherry picked from commit e4c9525b137205fa9ffdb4e0d7492bbbda9be6b5) (cherry picked from commit 09d7b83211652d045975b0e3bb790856267d52a5) (cherry picked from commit bbcd5975c91f6932f7f2ee07fbd63e84560ba96a) (cherry picked from commit 55c70a0e18d33d8ac0da9ffb97f6d994ed88a319) (cherry picked from commit 76596410c0dd0137cd497c9728c3e1d1c98f2430) (cherry picked from commit 1308043931388bf6de691ad0f766861b77fb08a5) (cherry picked from commit 919d6aedfed6abc8ec9def19f8deec2ee413252b) [FEAT] add Forgero Git Service (squash) more tests Previously only Gitea service was being tested under self-hosted migrations. Since Forgejo is also self-hosted and in fact use the same downloader/migrator we can add to this suite another test that will do the same, migrating the same repository under the same local instance but for the Forgejo service (represented by 9) Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/1709 Co-authored-by: zareck <cassiomilczareck@gmail.com> Co-committed-by: zareck <cassiomilczareck@gmail.com> (cherry picked from commit 40a4b8f1a8637f78cf2f48104f0b336377652df9)	8 months ago
wxiaoguang	6f9c278559	Rewrite queue (#24505) ... # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: 	1 year ago
KN4CK3R	a35749893b	Move `convert` package to services (#22264) ... Addition to #22256 The `convert` package relies heavily on different models which is [disallowed by our definition of modules](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#design-guideline). This helps to prevent possible import cycles. Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>	2 years ago