The current foundry pours for docker compose assume that the system memory is 32GB or greater. The compose (clickhouse container) will run and then spews out mergetree error after 2 or 3 days. my current system has 8GB and it is used for testing and development.
2026.09.08 07:28:39.745891 [ 646 ] {91a53a29-fd66-45b1-a648-b48fe4d80ffc::202609_42614_45484_46} <Error> virtual bool DB::MergePlainMergeTreeTask::executeStep(): Exception is in merge_task.: Code: 241. DB::Exception: (total) memory limit exceeded: would use 6.97 GiB (attempt to allocate chunk of 4.04 MiB), current RSS: 781.89 MiB, maximum: 6.97 GiB: (while reading column ProfileEvent_ReadBufferFromS3RequestsErrors): (while reading from part /var/lib/clickhouse/store/91a/91a53a29-fd66-45b1-a648-b48fe4d80ffc/202609_44954_45412_45/ in table system.metric_log (91a53a29-fd66-45b1-a648-b48fe4d80ffc) located on disk default of type local, from mark 0 with max_rows_to_read = 873, offset = 0): While reading part 202609_44954_45412_45: While executing MergeTreeSequentialSource. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):
0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000163b122a
1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000de0f24e
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000de0ec40
3. DB::Exception::Exception<char const*, char const*, String, String, String, String, String, char const*, std::basic_string_view<char, std::char_traits<char>>>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<char const*>::type, std::type_identity<std::basic_string_view<char, std::char_traits<char>>>::type>, char const*&&, char const*&&, String&&, String&&, String&&, String&&, String&&, char const*&&, std::basic_string_view<char, std::char_traits<char>>&&) @ 0x00000000164316b2
4. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642dc80
5. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
6. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
7. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
8. CurrentMemoryTracker::allocImpl(long, bool) @ 0x0000000016380512
9. Allocator<false, false>::alloc(unsigned long, unsigned long) @ 0x000000001637e924
10. DB::Memory<Allocator<false, false>>::alloc(unsigned long) @ 0x000000001649a5fe
11. DB::ReadBufferFromFileDescriptorPRead::ReadBufferFromFileDescriptorPRead(int, unsigned long, char*, unsigned long, std::optional<unsigned long>, std::shared_ptr<DB::IThrottler>) @ 0x00000000194d3b20
12. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, unsigned long)::$_1::operator()(unsigned long, unsigned long, int) const @ 0x00000000194d324e
13. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, unsigned long) @ 0x00000000194d2dc7
14. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001a5caf05
15. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001c555a19
16. DB::MergeTreeReaderStream::init() @ 0x000000001c8fa784
17. DB::MergeTreeReaderWide::getStream(bool, DB::ISerialization::SubstreamPath const&, DB::MergeTreeDataPartChecksums const&, DB::NameAndTypePair const&, unsigned long, bool, unsigned long, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>&) @ 0x000000001c9122a2
18. DB::ReadBuffer* std::__function::__policy_func<DB::ReadBuffer* (DB::ISerialization::SubstreamPath const&)>::__call_func[abi:ne210105]<DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, bool, unsigned long, unsigned long, unsigned long, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>&, std::unordered_map<String, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>>>>&)::$_0>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000001c914592
19. DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>*) const @ 0x000000001a2438d1
20. DB::SerializationSparse::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>*) const @ 0x000000001a3305c4
21. DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001c9107ee
22. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001c8e0499
23. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000001c8ea261
24. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&, std::vector<DB::MarkRanges, std::allocator<DB::MarkRanges>>&) @ 0x000000001c926b70
25. DB::MergeTreeSequentialSource::generate() @ 0x000000001c9197a6
26. DB::ISource::tryGenerate() @ 0x000000001d0872e9
27. DB::ISource::work() @ 0x000000001d086e81
28. DB::ExecutionThreadContext::executeTask() @ 0x000000001d0a4497
29. DB::PipelineExecutor::executeStepImpl(unsigned long, DB::IAcquiredSlot*, std::atomic<bool>*) @ 0x000000001d096104
30. DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000001d094ed8
31. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000001d0aeb05
(version 25.12.5.44 (official build))
Summary
The current foundry pours for docker compose assume that the system memory is 32GB or greater. The compose (clickhouse container) will run and then spews out mergetree error after 2 or 3 days. my current system has 8GB and it is used for testing and development.
Motivation
Allow for different memory configurations eg 8GB, 16GB, 32GB for clickhouse
Proposed solution
Allow foundry with max memory switch
Alternatives considered
Too much trouble to tune clickhouse config
Additional context
No response