unsigned long value overflow

Comments

• Matthias

  May 21

  If you have OASIS with large coordinates, you need to use an 64bit coordinate build. For Windows it's available as the "c64" binary. On Linux, you will need to build it yourself with the "-with-64bit-coord" option.

  By default, coordinates are 32 bit, which usually is more than enough. With 64 bit, the memory footprint roughly doubles, hence it's not default.

  But the error message is weird - I mean the negative position. Are you really using a 64bit version (not coordinates, 32bit binary)?

  Matthias
• Alexander

  May 22 edited May 22

  Thank you for your reply. I'm using a 64-bit binary with 32-bit coordinates. In the source code, I found this line:

  ReaderException(tl::sprintf(tl::to_string(tr("%s (position=%ld, cell=%s), in file: %s")), msg, p, cell, source));
  

  Here, p is a size_t, and I suspect that the issue might be caused by a cast from size_t (i.e., unsigned long) to %ld (signed long int ?).

  I'm now trying the 64-bit coordinate build and would also like to use the KLayout Python module. However, I couldn’t find the package on PyPI (no c64 package found). Could you please advise how to get it?

  I also noticed the 64-bit coordinate build includes a pymod folder — is that intended to be used as a Python module? It seems to be missing db_plugins.

  Thanks again!
• Alexander

  May 22 edited May 22

  I tried building pymod from source on Windows and added ("HAVE_64BIT_COORD", 1) to the macros section in setup.py, but it seems MSVC does not support __int128. Should i use msys2 instread ? or setup.py is used only for msvc2017? Thanks!
• Alexander

  May 24

  I found that this OAS file can be loaded successfully on Linux, but the same file causes an overflow error on Windows. Considering that the Linux version of KLayout was not built with --with-64bit-coord, could it mean that the issue is unrelated to this option?
• StefanThiede

  May 24 edited May 24

  It would help if you could either share the file or if you can’t, narrow the problem down til you can provide more info.
  Use klayout to make the file smaller while still failing.
  Try other tools that read oasis like https://github.com/wrcad/xictools and use https://github.com/klayoutmatthias/dump_oas_gds2 to have a look at the file’s contents.
• Matthias

  May 25

  Hi @Alexander,

  You don't need to build yourself on Windows - you can download the 64 bit coordinate version from the downloads page. See https://www.klayout.org/downloads/Windows/klayout-0.30.1-win64-c64-install.exe for example for 0.30.1. It is made with msys2. As you say, MSVC 2017 does not support __int128, but as it is mandatory for Windows modules, it's unlikely there will be 64bit coordinate support in Windows soon. I will also not be able to regularly deploy 64bit coordinate builds on Linux, as disk space is a scarce resource on PyPI.

  "pymod" is the Python module, but in order to run it, you need to set LD_LIBRARY_PATH, including "db_plugins". This module is not the same than on PyPI, as it includes Qt. It's not really in use currently. Better use setup.py - or better: the "build" module and "pyproject.toml". This will create a wheel for you.

  The "sprintf" issue will be solved, but it's likely not to be the cause of your problems. It just makes detection of the fault location more difficult.

  From your observation I think there is another issue. The Linux packages are not built with 64bit support. So maybe there is another issue. Like @StefanThiede said, a test case was helpful.

  There are few places where Windows would behave differently because of the "long int" interpretation (32 bit). That is IDs (text strings, property names, cell IDs) or repetitions (number of iterated instances, array dimensions). Maybe your design has such long IDs or giant arrays (also shape repetitions)?

  If you want to do serious debugging, you can try this little tool: https://github.com/klayoutmatthias/dump_oas_gds2

  Matthias
• Alexander

  September 18

  I believe I’ve identified the root cause of the issue. Some attributes appear to be file offsets, and when the file size exceeds 4 GB, these offsets may also exceed 4 GB. In the OAS parsing process, using unsigned long to parse variable-length numbers can therefore lead to overflow.

  To improve compatibility between Windows and Linux, I replaced unsigned long with size_t, which provides better cross-platform support. Below is my commit. If acceptable, I’d be happy to open a PR for merging.

  Since this issue is very important for my work, I would greatly appreciate it if you could help address it as soon as possible.
• StefanThiede

  September 18

  klayout fails to read CELLNAME S_CELL_OFFSET property on windows when it's value is > 4GB #2073
  This issue has been fixed with 0.30.3
  https://github.com/KLayout/klayout/issues/2073