dictionary - Memory usage of Dictionaries in C# -
i have code added nested dictionary to, of following format
dictionary<string, dictionary<string, dictionary<string, float>>> after doing noticed memory usage of application shot significantly. these dictionaries keyed on strings repeated, , there many of these dictionaries, on order of 10's of thousands.
in order address problem hypothesized repeated strings eating significant amount of memory. solution hash strings , use integer instead (i keep 1 copy of rainbow table reverse hash when necessary)
dictionary<int, dictionary<int, dictionary<int, float>>> so went memory profiler see kind of size reduction get. shock found string storage smaller in size (both normal , inclusive).
this doesn't make intuitive sense me. if compiler smart enough store 1 copy of string , use reference, think reference pointer double size of int. didn't use string.intern methods don't know how have been accomplished (also string.intern right method here?)
i'm confused what's happening under hood, appreciated
if keys , values objects, there's approximately 20 bytes of overhead each element of dictionary, plus several more bytes per dictionary. in addition space consumed keys , values themselves. if have value types keys , values, it's 12 bytes plus space consumed key , value each item in dictionary. if number of elements equals internal dictionary capacity. typically there more capacity elements, there wasted space.
the wasted space higher relative percentage if have lots of dictionaries small number of elements if had 1 dictionary many elements. if go comment, dictionaries 8 elements have capacity of 11, 2 elements have capacity of 3, , 10 have capacity of 11.
if understand nesting counts, single top level dictionary represent 184 dictionary elements. if count unused capacity, it's closer 200 far space consumption. 200 * 20 = 4000 bytes each top level dictionary. how many of have? 10's of thousands of them in thousand of objects. every 10,000 going consume 38 mb of dictionary overhead. add objects stored in dictionary.
a possible explanation of why attempt make smaller managing hash codes if there not lot of duplicated references keys. replacing object reference key int key doesn't change dictionary overhead amount, , you're adding storage of new collection of hash codes.
