Skip to content

[enhancement] Config for Memory Size #198

Description

@alberk8

Summary

The current foundry pours for docker compose assume that the system memory is 32GB or greater. The compose (clickhouse container) will run and then spews out mergetree error after 2 or 3 days. my current system has 8GB and it is used for testing and development.

2026.09.08 07:28:39.745891 [ 646 ] {91a53a29-fd66-45b1-a648-b48fe4d80ffc::202609_42614_45484_46} <Error> virtual bool DB::MergePlainMergeTreeTask::executeStep(): Exception is in merge_task.: Code: 241. DB::Exception: (total) memory limit exceeded: would use 6.97 GiB (attempt to allocate chunk of 4.04 MiB), current RSS: 781.89 MiB, maximum: 6.97 GiB: (while reading column ProfileEvent_ReadBufferFromS3RequestsErrors): (while reading from part /var/lib/clickhouse/store/91a/91a53a29-fd66-45b1-a648-b48fe4d80ffc/202609_44954_45412_45/ in table system.metric_log (91a53a29-fd66-45b1-a648-b48fe4d80ffc) located on disk default of type local, from mark 0 with max_rows_to_read = 873, offset = 0): While reading part 202609_44954_45412_45: While executing MergeTreeSequentialSource. (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below):

0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000163b122a
1. DB::Exception::Exception(String&&, int, String, bool) @ 0x000000000de0f24e
2. DB::Exception::Exception(PreformattedMessage&&, int) @ 0x000000000de0ec40
3. DB::Exception::Exception<char const*, char const*, String, String, String, String, String, char const*, std::basic_string_view<char, std::char_traits<char>>>(int, FormatStringHelperImpl<std::type_identity<char const*>::type, std::type_identity<char const*>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<char const*>::type, std::type_identity<std::basic_string_view<char, std::char_traits<char>>>::type>, char const*&&, char const*&&, String&&, String&&, String&&, String&&, String&&, char const*&&, std::basic_string_view<char, std::char_traits<char>>&&) @ 0x00000000164316b2
4. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642dc80
5. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
6. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
7. MemoryTracker::allocImpl(long, bool, MemoryTracker*, double) @ 0x000000001642d4b7
8. CurrentMemoryTracker::allocImpl(long, bool) @ 0x0000000016380512
9. Allocator<false, false>::alloc(unsigned long, unsigned long) @ 0x000000001637e924
10. DB::Memory<Allocator<false, false>>::alloc(unsigned long) @ 0x000000001649a5fe
11. DB::ReadBufferFromFileDescriptorPRead::ReadBufferFromFileDescriptorPRead(int, unsigned long, char*, unsigned long, std::optional<unsigned long>, std::shared_ptr<DB::IThrottler>) @ 0x00000000194d3b20
12. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, unsigned long)::$_1::operator()(unsigned long, unsigned long, int) const @ 0x00000000194d324e
13. DB::createReadBufferFromFileBase(String const&, DB::ReadSettings const&, std::optional<unsigned long>, std::optional<unsigned long>, int, char*, unsigned long) @ 0x00000000194d2dc7
14. DB::DiskLocal::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001a5caf05
15. DB::DataPartStorageOnDiskFull::readFile(String const&, DB::ReadSettings const&, std::optional<unsigned long>) const @ 0x000000001c555a19
16. DB::MergeTreeReaderStream::init() @ 0x000000001c8fa784
17. DB::MergeTreeReaderWide::getStream(bool, DB::ISerialization::SubstreamPath const&, DB::MergeTreeDataPartChecksums const&, DB::NameAndTypePair const&, unsigned long, bool, unsigned long, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>&) @ 0x000000001c9122a2
18. DB::ReadBuffer* std::__function::__policy_func<DB::ReadBuffer* (DB::ISerialization::SubstreamPath const&)>::__call_func[abi:ne210105]<DB::MergeTreeReaderWide::readData(DB::NameAndTypePair const&, std::shared_ptr<DB::ISerialization const> const&, COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, bool, unsigned long, unsigned long, unsigned long, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>&, std::unordered_map<String, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>>>>&)::$_0>(std::__function::__policy_storage const*, DB::ISerialization::SubstreamPath const&) @ 0x000000001c914592
19. DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>*) const @ 0x000000001a2438d1
20. DB::SerializationSparse::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, std::unique_ptr<DB::ISerialization::ISubstreamsCacheElement, std::default_delete<DB::ISerialization::ISubstreamsCacheElement>>>>>*) const @ 0x000000001a3305c4
21. DB::MergeTreeReaderWide::readRows(unsigned long, unsigned long, bool, unsigned long, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001c9107ee
22. DB::MergeTreeRangeReader::DelayedStream::finalize(std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001c8e0499
23. DB::MergeTreeRangeReader::startReadingChain(unsigned long, DB::MarkRanges&) @ 0x000000001c8ea261
24. DB::MergeTreeReadersChain::read(unsigned long, DB::MarkRanges&, std::vector<DB::MarkRanges, std::allocator<DB::MarkRanges>>&) @ 0x000000001c926b70
25. DB::MergeTreeSequentialSource::generate() @ 0x000000001c9197a6
26. DB::ISource::tryGenerate() @ 0x000000001d0872e9
27. DB::ISource::work() @ 0x000000001d086e81
28. DB::ExecutionThreadContext::executeTask() @ 0x000000001d0a4497
29. DB::PipelineExecutor::executeStepImpl(unsigned long, DB::IAcquiredSlot*, std::atomic<bool>*) @ 0x000000001d096104
30. DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x000000001d094ed8
31. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x000000001d0aeb05
 (version 25.12.5.44 (official build))

Motivation

Allow for different memory configurations eg 8GB, 16GB, 32GB for clickhouse

Proposed solution

Allow foundry with max memory switch

Alternatives considered

Too much trouble to tune clickhouse config

Additional context

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions