Need some help regarding Deep Learning/Python/TensorFlow
0
0
Entering edit mode
5.0 years ago
rbanaudha • 0

Hello All,

This is Ravindra, postdoc at NIH. Currently I am working in Deep learning/python things that is new for me. I posted this regarding some help in Deep learning, if someone can help me. I am learning deep learning using the script mentioned in the article "Predicting tumor cell line response to drug pairs with deep learning", that I cloned from the link (git clone https://github.com/ECP-CANDLE/Benchmarks.git).

1. For that, first I install miniconda at Biowulf (NIH HPC) using the following command

wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh

mkdir tmp

TMPDIR=$PWD/tmp bash Miniconda3-latest-Linux-x86_64.sh -p /data/$USER/conda -b

2.Then I created environment using my environment file that I created and run the following command

conda env create -f DNN_environment.yml

conda activate old_TF4

conda env list

Example of my environment file is

name: old_TF4

channels:

  • anaconda
  • conda-forge
  • defaults dependencies:
  • _libgcc_mutex=0.1=main
  • asn1crypto=0.24.0=py36_0
  • binutils_impl_linux-64=2.31.1=h6176602_1
  • binutils_linux-64=2.31.1=h6176602_8
  • blas=1.0=mkl
  • ca-certificates=2019.6.16=hecc5488_0
  • certifi=2019.6.16=py36_1
  • cffi=1.12.3=py36h2e261b9_0
  • chardet=3.0.4=py36_1003
  • cryptography=2.7=py36h1ba5d50_0
  • cycler=0.10.0=py36_0
  • dbus=1.13.6=h746ee38_0
  • expat=2.2.6=he6710b0_0
  • fontconfig=2.13.0=h9420a91_0
  • freetype=2.9.1=h8a8886c_1
  • gcc_impl_linux-64=7.3.0=habb00fd_1
  • gcc_linux-64=7.3.0=h553295d_8
  • glib=2.56.2=hd408876_0
  • gst-plugins-base=1.14.0=hbbd80ab_1
  • gstreamer=1.14.0=hb453b48_1
  • gxx_impl_linux-64=7.3.0=hdf63c60_1
  • gxx_linux-64=7.3.0=h553295d_8
  • h5py=2.7.0=np111py36_0
  • hdf5=1.8.17=11
  • icu=58.2=h9c2bf20_1
  • idna=2.8=py36_0
  • intel-openmp=2019.4=243
  • joblib=0.13.2=py36_0
  • jpeg=9b=h024ee3a_2
  • keras=2.2.4=0
  • keras-applications=1.0.8=py_0
  • keras-base=2.2.4=py36_0
  • keras-preprocessing=1.1.0=py_1
  • kiwisolver=1.1.0=py36he6710b0_0
  • libedit=3.1.20181209=hc058e9b_0
  • libffi=3.2.1=hd88cf55_4
  • libgcc-ng=9.1.0=hdf63c60_0
  • libgfortran=3.0.0=1
  • libgfortran-ng=7.3.0=hdf63c60_0
  • libgpuarray=0.7.6=h14c3975_0
  • libpng=1.6.37=hbc83047_0
  • libprotobuf=3.8.0=hd408876_0
  • libstdcxx-ng=9.1.0=hdf63c60_0
  • libuuid=1.0.3=h1bed415_2
  • libxcb=1.13=h1bed415_1
  • libxml2=2.9.9=hea5a465_1
  • mako=1.0.10=py_0
  • markupsafe=1.1.1=py36h7b6447c_0
  • matplotlib=3.1.0=py36_0
  • matplotlib-base=3.1.0=py36h5f35d83_0
  • mkl=2019.4=243
  • mkl-service=2.0.2=py36h7b6447c_0
  • mock=3.0.5=py36_0
  • ncurses=6.1=he6710b0_1
  • numpy=1.11.3=py36hcd700cb_6
  • numpy-base=1.16.4=py36hde5b4d6_0
  • openssl=1.1.1c=h516909a_0
  • pandas=0.20.1=np111py36_0
  • pcre=8.43=he6710b0_0
  • pip=19.2.2=py36_0
  • protobuf=3.8.0=py36he6710b0_0
  • pycparser=2.19=py36_0
  • pygpu=0.7.6=py36h035aef0_0
  • pyopenssl=19.0.0=py36_0
  • pyparsing=2.4.2=py_0
  • pyqt=5.9.2=py36h05f1152_2
  • pysocks=1.7.0=py36_0
  • python=3.6.9=h265db76_0
  • python-dateutil=2.8.0=py36_0
  • pytz=2019.2=py_0
  • pyyaml=5.1.2=py36h7b6447c_0
  • qt=5.9.7=h5867ecd_1
  • readline=7.0=h7b6447c_5
  • requests=2.22.0=py36_0
  • scikit-learn=0.21.2=py36hd81dba3_0
  • scipy=1.3.1=py36h7c811a0_0
  • setuptools=41.0.1=py36_0
  • sip=4.19.8=py36hf484d3e_0
  • six=1.12.0=py36_0
  • sqlite=3.29.0=h7b6447c_0
  • tensorflow=1.0.0=py36_0
  • theano=1.0.3=py36hfd86e86_0
  • tk=8.6.9=hed695b0_1002
  • tornado=6.0.3=py36h7b6447c_0
  • tqdm=4.35.0=py_0
  • urllib3=1.24.2=py36_0
  • wheel=0.33.4=py36_0
  • xz=5.2.4=h14c3975_4
  • yaml=0.1.7=had09818_2
  • zlib=1.2.11=h7b6447c_3

3. I coned the repository from the link (git clone https://github.com/ECP-CANDLE/Benchmarks.git) and train the model using the following command

cd Benchmark/Pilot1/Combo

python combo_baseline_keras2.py

The above command run successfully.

4. But when I try to run validation by following command using their model that I downloaded from the link that they provided in Benchmark/Pilot1/Combo/README.md (http://ftp.mcs.anl.gov/pub/candle/public/benchmarks/Pilot1/combo/saved.model.h5 and http://ftp.mcs.anl.gov/pub/candle/public/benchmarks/Pilot1/combo/saved.weights.h5)

python infer.py

It could not run successfully and shows some error that looks like

/data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype fromfloattonp.floatingis deprecated. In future, it will be treated asnp.float64 == np.dtype(float).type`. from ._conv import register_converters as _register_converters

Using TensorFlow backend. /data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/sklearn/utils/deprecation.py:66: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead. warnings.warn(msg, category=DeprecationWarning) /data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/sklearn/utils/deprecation.py:66: DeprecationWarning: Class Imputer is deprecated; Imputer was deprecated in version 0.20 and will be removed in 0.22. Import impute.SimpleImputer from sklearn instead. warnings.warn(msg, category=DeprecationWarning)

0%| | 0/1296 [00:00 run()

File "infer.py", line 196, in run

y_pred = model.predict(x_all_list, batch_size=args.batch_size, verbose=0).flatten()

File "/data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/keras/engine/training.py", line 1149, in predict

x, _, _ = self._standardize_user_data(x)

File "/data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/keras/engine/training.py", line 751, in _standardize_user_data exception_prefix='input')

File "/data/kumarr8/conda/envs/old_TF4/lib/python3.6/site-packages/keras/engine/training_utils.py", line 138, in standardize_input_data str(data_shape))

ValueError: Error when checking input: expected input.cell.expression to have shape (942,) but got array with shape (17743,)`

Also If I wants to change my input file for validation, how can I change it?

Could anyone please help me to resolve the issue?

Thanks in advance!!!

Ravindra
Deep Learning Python TensorFlow • 1.3k views
ADD COMMENT •
0
Entering edit mode

"ValueError: Error when checking input: expected input.cell.expression to have shape (942,) but got array with shape (17743,)".

My guess is that the model only used 942 genes for training, and here you provided 17743 genes for prediction, which caused this input error.

ADD REPLY •
0
Entering edit mode

Hi Shoujun,

Thanks for the suggestion, I am working on this but I don't know how to resolve it, since I'm new in python.

BTW thanks again, Ravindra

ADD REPLY •

Login before adding your answer.

Similar Posts
Loading Similar Posts
Traffic: 1658 users visited in the last hour
Content Search
Users
Tags
Badges
Help About
FAQ
Access RSS
API
Stats

Use of this site constitutes acceptance of our User Agreement and Privacy Policy.

Powered by the version 2.3.6