「Pythonではじめるオープンエンドな進化的アルゴリズム」の出版に関わらせていただいた。この書籍の中では、 Evolution Gym
というツールを使って実装例を示している。 Evolution Gym
は良くできたツールなのだが、インストールに失敗するケースがいくつかあった。こちらで認識できているものについては本家にプルリクエストを送った。しかし想定していない問題があるようだった。
その問題とは h5py
のインストール時に実行されるビルドに失敗するという物だった。私が使っている環境ではこの問題に気付く事ができなかった。過去のログなどを確認すると、whl形式のパッケージをダウンロードしていたようだった。whlの中にはビルド済みバイナリが含まれている。そのためビルドを実行せずインストールに成功していたようだ。
そこで今回は h5py
をwhlを使わずインストールする事にした。
hdf5をインストールする
Cythonをインストールする
h5py
のビルドには Cython
を使うため、 Cython
をインストールする。インストールの様子はhttps://blog.symdon.info/posts/1697872568/にまとめた。
h5py-3.6.0のインストールで右往左往する
Evolution Gym
は h5py-3.6.0
を requirements.txt
に含めている。まず、このバージョンを pip
でインストールできるかを確認する事にした。 h5py-3.6.0
では以下のwhl形式が提供されている3。
- h5py-3.6.0-cp310-cp310-macosx_10_9_x86_64.whl
- h5py-3.6.0-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
- h5py-3.6.0-cp310-cp310-win_amd64.whl
- h5py-3.6.0-cp37-cp37m-macosx_10_9_x86_64.whl
- h5py-3.6.0-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
- h5py-3.6.0-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- h5py-3.6.0-cp37-cp37m-win_amd64.whl
- h5py-3.6.0-cp38-cp38-macosx_10_9_x86_64.whl
- h5py-3.6.0-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
- h5py-3.6.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- h5py-3.6.0-cp38-cp38-win_amd64.whl
- h5py-3.6.0-cp39-cp39-macosx_10_9_x86_64.whl
- h5py-3.6.0-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
- h5py-3.6.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- h5py-3.6.0-cp39-cp39-win_amd64.whl
インストールする環境がこの範囲に含まれていれば、whlを使ってインストールされる。
pip install 'h5py==3.6.0'
Collecting h5py==3.6.0 Downloading h5py-3.6.0-cp38-cp38-macosx_10_9_x86_64.whl (3.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.1/3.1 MB 331.7 kB/s eta 0:00:00 Requirement already satisfied: numpy>=1.14.5 in /Users/foo/.venv/testing-h5py/lib/python3.8/site-packages (from h5py==3.6.0) (1.21.5) Installing collected packages: h5py Attempting uninstall: h5py Found existing installation: h5py 3.10.0 Uninstalling h5py-3.10.0: Successfully uninstalled h5py-3.10.0 Successfully installed h5py-3.6.0
whlを用いてインストールされた。以降の確認のため一度アンインストールする。
pip uninstall -y h5py
Found existing installation: h5py 3.10.0 Uninstalling h5py-3.10.0: Successfully uninstalled h5py-3.10.0
先ほど一覧でしめしたwhlの対応する環境に当てはまらなければ、tarballをダウンロードしビルドされる事になる。例えばM1のmacOSを使っている場合には、ビルドが行われる。whlを使用したくない場合は pip install
のオプションとして --no-binary\=h5py
を指定する。
pip install 'h5py==3.6.0' --no-binary=h5py 2>&1
Collecting h5py==3.6.0 Using cached h5py-3.6.0.tar.gz (384 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Requirement already satisfied: numpy>=1.14.5 in /Users/foo/.venv/testing-h5py/lib/python3.8/site-packages (from h5py==3.6.0) (1.21.5) Building wheels for collected packages: h5py Building wheel for h5py (pyproject.toml): started Building wheel for h5py (pyproject.toml): finished with status 'error' error: subprocess-exited-with-error × Building wheel for h5py (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [528 lines of output] running bdist_wheel running build running build_py creating build creating build/lib.macosx-13-x86_64-cpython-38 creating build/lib.macosx-13-x86_64-cpython-38/h5py copying h5py/h5py_warnings.py -> build/lib.macosx-13-x86_64-cpython-38/h5py copying h5py/version.py -> build/lib.macosx-13-x86_64-cpython-38/h5py copying h5py/__init__.py -> build/lib.macosx-13-x86_64-cpython-38/h5py copying h5py/ipy_completer.py -> build/lib.macosx-13-x86_64-cpython-38/h5py creating build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/files.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/compat.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/__init__.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/selections.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/dataset.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/vds.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/selections2.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/group.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/datatype.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/attrs.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/dims.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/base.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl copying h5py/_hl/filters.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/_hl creating build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dimension_scales.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_attribute_create.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_file_image.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/conftest.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5d_direct_chunk.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5f.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dataset_getitem.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_group.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_errors.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dataset_swmr.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_slicing.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5pl.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_attrs.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/__init__.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_attrs_data.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5t.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_big_endian_file.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5p.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dims_dimensionproxy.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5o.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_datatype.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/common.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dataset.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_file.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_selections.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_dtype.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_h5.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_file2.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_completions.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_filters.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_base.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests copying h5py/tests/test_objects.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests creating build/lib.macosx-13-x86_64-cpython-38/h5py/tests/data_files copying h5py/tests/data_files/__init__.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/data_files creating build/lib.macosx-13-x86_64-cpython-38/h5py/tests/test_vds copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/test_vds copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/test_vds copying h5py/tests/test_vds/__init__.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/test_vds copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/test_vds copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/data_files copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/data_files copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.macosx-13-x86_64-cpython-38/h5py/tests/data_files running build_ext warning: h5py/defs.pxd:15:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:62:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:64:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:66:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:68:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:70:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:72:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:74:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:76:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:78:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:102:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:104:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:106:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:107:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:109:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:110:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:112:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:114:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:116:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:180:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:197:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:199:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:201:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:203:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:213:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:215:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:229:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:231:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:233:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:235:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:237:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:239:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:241:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:269:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:271:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:273:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:275:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:277:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:279:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:285:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:287:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:289:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:291:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:293:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:295:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:297:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:299:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:333:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:335:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:337:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:339:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:341:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:343:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:345:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:347:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:349:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:383:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/defs.pxd:385:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:48:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:64:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:143:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:148:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:156:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:165:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:198:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:287:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_hdf5.pxd:334:2: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_ext.pxd:14:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_ext.pxd:25:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/api_types_ext.pxd:52:4: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/_conv.pyx:161:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/_conv.pyx:422:8: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 warning: h5py/h5r.pxd:17:0: The 'IF' statement is deprecated and will be removed in a future Cython version. Consider using runtime conditions or C macros instead. See https://github.com/cython/cython/issues/4310 Error compiling Cython file: ------------------------------------------------------------ ... # License: Standard 3-clause BSD; see "license.txt" for full license terms # and contributor agreement. from .defs cimport * from ._objects cimport class ObjectID ^ ------------------------------------------------------------ h5py/h5t.pxd:13:23: Expected an identifier Error compiling Cython file: ------------------------------------------------------------ ... from logging import getLogger from .h5 import get_config from .h5r cimport Reference, RegionReference, hobj_ref_t, hdset_reg_ref_t from .h5t cimport H5PY_OBJ, typewrap, py_create, TypeID, H5PY_PYTHON_OPAQUE_TAG ^ ------------------------------------------------------------ h5py/_conv.pyx:21:0: 'h5py/h5t/H5PY_OBJ.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... from logging import getLogger from .h5 import get_config from .h5r cimport Reference, RegionReference, hobj_ref_t, hdset_reg_ref_t from .h5t cimport H5PY_OBJ, typewrap, py_create, TypeID, H5PY_PYTHON_OPAQUE_TAG ^ ------------------------------------------------------------ h5py/_conv.pyx:21:0: 'h5py/h5t/typewrap.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... from logging import getLogger from .h5 import get_config from .h5r cimport Reference, RegionReference, hobj_ref_t, hdset_reg_ref_t from .h5t cimport H5PY_OBJ, typewrap, py_create, TypeID, H5PY_PYTHON_OPAQUE_TAG ^ ------------------------------------------------------------ h5py/_conv.pyx:21:0: 'h5py/h5t/py_create.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... from logging import getLogger from .h5 import get_config from .h5r cimport Reference, RegionReference, hobj_ref_t, hdset_reg_ref_t from .h5t cimport H5PY_OBJ, typewrap, py_create, TypeID, H5PY_PYTHON_OPAQUE_TAG ^ ------------------------------------------------------------ h5py/_conv.pyx:21:0: 'h5py/h5t/TypeID.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... from logging import getLogger from .h5 import get_config from .h5r cimport Reference, RegionReference, hobj_ref_t, hdset_reg_ref_t from .h5t cimport H5PY_OBJ, typewrap, py_create, TypeID, H5PY_PYTHON_OPAQUE_TAG ^ ------------------------------------------------------------ h5py/_conv.pyx:21:0: 'h5py/h5t/H5PY_PYTHON_OPAQUE_TAG.pxd' not found Error compiling Cython file: ------------------------------------------------------------ ... void* ptr cdef int conv_vlen2ndarray(void* ipt, void* opt, cnp.dtype elem_dtype, TypeID intype, ^ ------------------------------------------------------------ h5py/_conv.pyx:687:27: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... cdef int conv_vlen2ndarray(void* ipt, void* opt, cnp.dtype elem_dtype, TypeID intype, TypeID outtype) except -1: ^ ------------------------------------------------------------ h5py/_conv.pyx:688:27: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... return 0 cdef int conv_ndarray2vlen(void* ipt, void* opt, TypeID intype, ^ ------------------------------------------------------------ h5py/_conv.pyx:820:27: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... cdef int conv_ndarray2vlen(void* ipt, void* opt, TypeID intype, TypeID outtype) except -1: ^ ------------------------------------------------------------ h5py/_conv.pyx:821:27: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... :return: error-code """ cdef: int command = cdata[0].command size_t src_size, dst_size TypeID supertype ^ ------------------------------------------------------------ h5py/_conv.pyx:629:8: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... """ cdef: int command = cdata[0].command size_t src_size, dst_size TypeID supertype TypeID outtype ^ ------------------------------------------------------------ h5py/_conv.pyx:630:8: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... void *bkg_i, hid_t dxpl) except -1 with gil: cdef: int command = cdata[0].command size_t src_size, dst_size TypeID supertype ^ ------------------------------------------------------------ h5py/_conv.pyx:752:8: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... hid_t dxpl) except -1 with gil: cdef: int command = cdata[0].command size_t src_size, dst_size TypeID supertype TypeID outtype ^ ------------------------------------------------------------ h5py/_conv.pyx:753:8: 'TypeID' is not a type identifier Error compiling Cython file: ------------------------------------------------------------ ... cdef char* ctag = NULL try: if H5Tget_class(obj) == H5T_OPAQUE: ctag = H5Tget_tag(obj) if ctag != NULL: if strcmp(ctag, H5PY_PYTHON_OPAQUE_TAG) == 0: ^ ------------------------------------------------------------ h5py/_conv.pyx:157:32: 'H5PY_PYTHON_OPAQUE_TAG' is not a constant, variable or function identifier warning: h5py/_conv.pyx:157:32: Obtaining 'const char *' from externally modifiable global Python value Error compiling Cython file: ------------------------------------------------------------ ... elif command == H5T_CONV_FREE: pass elif command == H5T_CONV_CONV: # need to pass element dtype to converter supertype = typewrap(H5Tget_super(src_id)) ^ ------------------------------------------------------------ h5py/_conv.pyx:645:20: 'typewrap' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... elif command == H5T_CONV_CONV: # need to pass element dtype to converter supertype = typewrap(H5Tget_super(src_id)) dt = supertype.dtype outtype = py_create(dt) ^ ------------------------------------------------------------ h5py/_conv.pyx:647:18: 'py_create' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... PyObject *pdata_elem char* buf = <char*>buf_i if command == H5T_CONV_INIT: cdata[0].need_bkg = H5T_BKG_NO if not H5Tequal(src_id, H5PY_OBJ) or H5Tget_class(dst_id) != H5T_VLEN: ^ ------------------------------------------------------------ h5py/_conv.pyx:761:32: 'H5PY_OBJ' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... if command == H5T_CONV_INIT: cdata[0].need_bkg = H5T_BKG_NO if not H5Tequal(src_id, H5PY_OBJ) or H5Tget_class(dst_id) != H5T_VLEN: return -2 supertype = typewrap(H5Tget_super(dst_id)) ^ ------------------------------------------------------------ h5py/_conv.pyx:763:20: 'typewrap' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... return -2 supertype = typewrap(H5Tget_super(dst_id)) for i in range(nl): # smells a lot memcpy(&pdata_elem, pdata+i, sizeof(pdata_elem)) if supertype != py_create((<cnp.ndarray> pdata_elem).dtype, 1): ^ ------------------------------------------------------------ h5py/_conv.pyx:767:28: 'py_create' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... if nl == 0: return 0 # need to pass element dtype to converter pdata_elem = pdata[0] supertype = py_create((<cnp.ndarray> pdata_elem).dtype) ^ ------------------------------------------------------------ h5py/_conv.pyx:784:20: 'py_create' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... return 0 # need to pass element dtype to converter pdata_elem = pdata[0] supertype = py_create((<cnp.ndarray> pdata_elem).dtype) outtype = typewrap(H5Tget_super(dst_id)) ^ ------------------------------------------------------------ h5py/_conv.pyx:785:18: 'typewrap' is not a constant, variable or function identifier Error compiling Cython file: ------------------------------------------------------------ ... enum = H5Tenum_create(H5T_STD_I32LE) vlentype = H5Tvlen_create(H5T_STD_I32LE) pyobj = H5PY_OBJ ^ ------------------------------------------------------------ h5py/_conv.pyx:894:12: 'H5PY_OBJ' is not a constant, variable or function identifier Loading library to get build settings and version: libhdf5.dylib ******************************************************************************** Summary of the h5py configuration HDF5 include dirs: [] HDF5 library dirs: [] HDF5 Version: (1, 14, 2) MPI Enabled: False ROS3 VFD Enabled: False Rebuild Required: True ******************************************************************************** Executing api_gen rebuild of defs Executing cythonize() [ 1/24] Cythonizing /private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-install-8b6c2mrb/h5py_3a9bffda4ab548d5ac5aea13ae9c6ee1/h5py/_conv.pyx Traceback (most recent call last): File "/Users/foo/.venv/testing-h5py/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module> main() File "/Users/foo/.venv/testing-h5py/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) File "/Users/foo/.venv/testing-h5py/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel return _build_backend().build_wheel(wheel_directory, config_settings, File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 434, in build_wheel return self._build_with_temp_dir( File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 419, in _build_with_temp_dir self.run_setup() File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 341, in run_setup exec(code, locals()) File "<string>", line 104, in <module> File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup return distutils.core.setup(**attrs) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands self.run_command(cmd) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command super().run_command(command) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 364, in run self.run_command("build") File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command super().run_command(command) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run self.run_command(cmd_name) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command self.distribution.run_command(command) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command super().run_command(command) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command cmd_obj.run() File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-install-8b6c2mrb/h5py_3a9bffda4ab548d5ac5aea13ae9c6ee1/setup_build.py", line 170, in run self.extensions = cythonize(self._make_extensions(config), File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize cythonize_one(*args) File "/private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-build-env-uvpwfguv/overlay/lib/python3.8/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one raise CompileError(None, pyx_file) Cython.Compiler.Errors.CompileError: /private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-install-8b6c2mrb/h5py_3a9bffda4ab548d5ac5aea13ae9c6ee1/h5py/_conv.pyx [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building wheel for h5py Failed to build h5py ERROR: Could not build wheels for h5py, which is required to install pyproject.toml-based projects WARNING: There was an error checking the latest version of pip.
from ~ cimport class ~
構文が使用できなかったり、インポートする対象が存在しなかったりと、よく分からない状態になっていた。ログを見るとCythonのバージョンが正しくないといった出力もある。
構文の問題については、Cython-3.0.0のあたりで修正されたもののようだった。
The undocumented, untested and apparently useless syntax from somemodule cimport class/struct/union somename was removed. The type modifier is not needed here and a plain cimport of the name will do. (Github issue #4904)
issue(https://github.com/cython/cython/pull/4904)もある。tarballの中身を確認すると、確かに pyproject.toml
には Cython-0.29.14
を要求する記述があった。
[build-system]
# Further build requirements - like numpy & cython - come from setup.py via
# the PEP 517 interface.
requires = [
"setuptools",
"wheel",
"oldest-supported-numpy",
"pkgconfig",
"Cython >=0.29; python_version<'3.8'",
"Cython >=0.29.14; python_version=='3.8'",
"Cython >=0.29.15; python_version>='3.9'",
]
build-backend = "setuptools.build_meta"
そこで幾つかの Cython
のバージョンを確認する事にした。要求されていないものについても、もしかしたらと思い確認したが、結果としては全て同じエラーになった。
cython-3.0.4
cython-0.29.14
cython-0.29.13
cython-0.29
cython-0.28
whl
として存在している h5py
はどのようにビルドされたのか、結局よく分からなかった。ただ、これ以上は深追いせず、 h5py==3.6.0
のソースコードからのビルドを諦める事にした。本当にやりたい事は「 Evolution Gym
を使いたい」だけであって、 h5py==3.6.0
をソースコードからビルドしたい訳ではないからだった。
Evolution Gymはh5pyを使っているのか?
h5py==3.6.0
のソースコードからのビルドを諦めたため、一度立ち止まって別の方向を考える事にした。元々は Evolution Gym
の requirements.txt
にバージョン指定で記述されている事が発端だった。もし Evolution Gym
で h5py
で直接 import
していないのであれば、そもそも requirements.txt
には記述する必要はない。
そこで Evolution Gym
のコードの中で h5py
の記述がある場所を grep
した。
$ find . -type f -exec grep --color=auto -nH --null -e h5py \{\} + ./requirements.txt:5:h5py==3.6.0 ./environment.yml:12: - h5py==3.6.0 ./examples/externals/pytorch_a2c_ppo_acktr_gail/a2c_ppo_acktr/algo/gail.py:1:import h5py ./examples/externals/pytorch_a2c_ppo_acktr_gail/requirements.txt:5:h5py ./examples/externals/pytorch_a2c_ppo_acktr_gail/gail_experts/convert_to_pytorch.py:5:import h5py ./examples/externals/pytorch_a2c_ppo_acktr_gail/gail_experts/convert_to_pytorch.py:28: with h5py.File(args.h5_file, 'r') as f: Grep exited abnormally with code 1 at Sun Oct 22 08:21:29, duration 4.08 s
h5py
は使われていたが、使われている箇所は限定的だった。しかもよく見ると git submodule
として登録されている、 examples/externals/pytorch_a2c_ppo_acktr_gail
で使用しているだけだった。しかも examples
ディレクトリは実装例のコードがあるだけであって、 Evolution Gym
本体ではない。つまり Evolution Gym
自身は h5py
を必要とせず、実装例を実行する場合に必要になるという事だった。
h5py
をuninstallした状態で Evolution Gym
のシミュレータを起動してみる事にした。
pyhton examples/gym_test.py
シミュレータは起動できた。これだけでは Evolution Gym
全体で正しく動作するとは言えないけれど、少なくとも Evolution Gym
の requirements.txt
には記述しなくても良さそうだった。
h5py-3.10.0をインストールする
h5py
はインストールしなくても良さそうだったが、インストールできないままでは目覚めが悪い。そこで最新の h5py
を入れてみる事にした。最新のバージョンは現時点で h5py-3.10.0
だった4。
pip install h5py --no-binary=h5py --no-cache
Found existing installation: h5py 3.10.0 Uninstalling h5py-3.10.0: Would remove: /Users/foo/.venv/testing-h5py/lib/python3.8/site-packages/h5py-3.10.0.dist-info/* /Users/foo/.venv/testing-h5py/lib/python3.8/site-packages/h5py/* Proceed (Y/n)? Y Successfully uninstalled h5py-3.10.0 bash-5.2$ pip install h5py --no-binary=h5py --no-cache Collecting h5py Downloading h5py-3.10.0.tar.gz (403 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 403.7/403.7 kB 1.1 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: numpy>=1.17.3 in /Users/foo/.venv/testing-h5py/lib/python3.8/site-packages (from h5py) (1.21.5) Building wheels for collected packages: h5py Building wheel for h5py (pyproject.toml) ... - done Created wheel for h5py: filename=h5py-3.10.0-cp38-cp38-macosx_13_0_x86_64.whl size=1483983 sha256=12cca7e0eebba95f877a86dbfe7db5929870efa700feb274035b36e44f100012 Stored in directory: /private/var/folders/2f/1mkpnp3n1gjd0r7l9nm_dyhr0000gn/T/pip-ephem-wheel-cache-zu_e4nqr/wheels/24/36/b8/d4f87683938866423e3ad9c8d3cfd788089776585856660d2f Successfully built h5py Installing collected packages: h5py Successfully installed h5py-3.10.0
再現できなくなってしまったけれど遭遇した問題
作業をしている最中に遭遇したが、再現させようとしてできなくなってしまった問題を掲載しておく。内容が Cython
と h5py
のインストール時でごちゃまぜになってしまっている。一部(もしくは全部)は Cython
の話をしているかもしれない。
gfortran
gfortran
がないエラーに遭遇した。これは Homebrew
で素直に gfortran
をインストールした。
MACOSX_DEPLOYMENT_TARGET環境変数
MACOSX_DEPLOYMENT_TARGET
環境変数が設定されてないと、このバージョンの値がintとなり、strを期待している箇所との食い違いによりエラーが発生する事があった。とりあえず使っているmacOSのバージョンを調べ MACOSX_DEPLOYMENT_TARGET
環境変数に値を設定したところ、問題は解消した。
export MACOSX_DEPLOYMENT_TARGET=13.6