I need to produce large and big (very) matrices (Markov chains) for scientific purposes. I perform calculus that I put in a list of 20301 elements (=one row of my matrix). I need all those data in memory to proceed next Markov step but i can store them elsewhere (eg file) if needed even if it will slow my Markov chain walk-through. My computer (scientific lab): Bi-xenon 6 cores/12threads each, 12GB memory, OS: win64

  Traceback (most recent call last):
  File "my_file.py", line 247, in <module>
    ListTemp.append(calculus)
MemoryError

Example of calculus results: 9.233747520008198e-102 (yes, it's over 1/9000)

The error is raised when storing the 19766th element:

ListTemp[19766]
1.4509421012263216e-103

If I go further

Traceback (most recent call last):
  File "<pyshell#21>", line 1, in <module>
    ListTemp[19767]
IndexError: list index out of range

So this list had a memory error at the 19767 loop.

Questions:

1. Is there a memory limit to a list? Это ограничение по списку или "глобальный лимит на скрипт"?

2. Как обойти эти ограничения? Есть ли какие-нибудь возможности?

3. Поможет ли использование numpy, python64? какой лимиты памяти с ними? какой о других языках?