Having problems at step 2

[6wom9]

I cloned this code from Github according to requirements.txt. And then, I've run "shrink_data.py". But I am having problems at step 2. What should I do in this condition? Although I've checked this issue, I can not understand anything.

tensorflow == 1.15.0

I'm very very new with python.

[di-press]

Hi, @6wom9 !

I followed the README instructions and ended up with the same error as yours...
Couldn't fix it yet

[saminfatehi]

Hmm... that's strange. Did you guys download the Bert model file and decompress it?

[di-press]

Hi @saminfatehir!

I decompressed the folder inside the personality_detection project folder, like this:

The Bert model I downloaded was " BERT-Base, Uncased | 12-layer, 768-hidden, 12-heads, 110M parameters"

Thanks for your help!

[saminfatehi]

At first please double-check the model path (model_dir).
Second, did you check this issue? Maybe the problem is about your tensorflow version.

[di-press]

Hi, @saminfatehir!
I can run the code now, and the problem was the tensorflow version, as you pointed out.
I was using tensorflow 2.4.1, and I needed to downgrade the version to tensorflow to 1.13.0

Thanks, @saminfatehir ! :)

@6wom9 I could run the code using Python 3.7. To fix the problem, try these steps:

1- Uninstall your previous tensorflow version (try in the terminal: pip uninstall tensorflow and see if it works)
2- Install an oldest version of tensorflow, by typing in the terminal: pip install tensorflow==1.13.0rc1
3- Be sure you have downloaded the Bert model (in my case, I downloaded "Bert base uncased")
4- Check if you decompressed the folder correctly by checking if your project folder hierarchy is organized as shown in the picture I posted in this issue before.
5- Try the step 2 suggested in the README

Did it work?

[saminfatehi]

Thanks for your description @di-press;
If this is helpful, you can make us glad by giving our project a star. :D

[6wom9]

Hi, @saminfatehir!
I can run the code now, and the problem was the tensorflow version, as you pointed out.
I was using tensorflow 2.4.1, and I needed to downgrade the version to tensorflow to 1.13.0

Thanks, @saminfatehir ! :)

@6wom9 I could run the code using Python 3.7. To fix the problem, try these steps:

1- Uninstall your previous tensorflow version (try in the terminal: pip uninstall tensorflow and see if it works)
2- Install an oldest version of tensorflow, by typing in the terminal: pip install tensorflow==1.13.0rc1
3- Be sure you have downloaded the Bert model (in my case, I downloaded "Bert base uncased")
4- Check if you decompressed the folder correctly by checking if your project folder hierarchy is organized as shown in the picture I posted in this issue before.
5- Try the step 2 suggested in the README

Did it work?

Hi @di-press, thanks for your great contributions.

After implying your advices, step 2 and step 3 worked well.

However at step 4, I am still having some problems.

I run the IDE (Pycharm) run as administrator, but It's not worked.

[di-press]

Hi @6wom9!

I coulnd't run the code completely yet. I'm having an error at step 2, and couldn't complete all the steps suggested in the README. It seems I'm having a problem with bert-as-service. When I type in the terminal, as suggested in this README in step 2, this command:

bert-serving-start -model_dir uncased_L-12_H-768_A-12/ -num_worker=4 -max_seq_len=NONE -show_tokens_to_client -pooling_layer -12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1

my computer crashes instantly, and I need to reboot it manually. It crashes when WORKER-1 tries to load a graph (do you remember in step 2, where the workers appears, in terminal, trying to load graphs? It is in this step that I have an issue). Maybe I don't have enough memory or other hardrware requirement to run the code... Please, can you tell:

1- how much memory you have?
2- if bert-as-service could use your GPU?

Really thanks!
If later I can complete all the steps between 2 and 4, I probably will know how to help you better...

[6wom9]

Hi @saminfatehir,
I want to complete process and give a star your project so I'm still trying to overcome bugs. I try to run this code for a paper. IF I manage to run this code, I will also cite your paper too. I shared a couple of screenshots which included some error messages. Although "svm_0_layer_with_mairesse_rev200_.csv" was generated but it was empty.

Hi again @di-press

I also couldn't run the code with my personal computer, It crashed at step 2. For this reason I am using computer laboratory in my university.
32 gb RAM
Intel(R) Xeon(R) W-2133 CPU @ 3.60GHz

[saminfatehi]

Hi @6wom9

Hmmm... That's strange.
what is your sklearn version? maybe an upgrade could be helpful.

[HeptanesianVibrionic]

Good morning @saminfatehir,

Apologies for any ignorance, as I'm relatively new to Python.

I completed step 1. Upon step two, my terminal in both PyCharm and Command Prompt seems to stagnate indefinitely at the point in the screenshot above.

As per the above comments, I have ensured that the BERT model is decompressed directly into the personality_detection folder, using the same BERT file used in the example, such that copying and pasting the exact example line for step 2 seems to access it fine.

I also depreciated TensorFlow to 1.13.0rc0, 1.13.0rc1, and 1.13.0rc2, all to the same effect. I am currently using 1.13.0rc1 as I attempt other solutions.

Please don't let me know if there's any additional information you would need for this.

Computer specifications:

• Device: HP ENVY x360 Convertible 15-dr1
• Processor: Intel(R) Core(TM) i5-10210U CPU @ 1.60 GHz 2.11 GHz
• Installed RAM: 8.00 GB
• 64-bit operating system, x64-based processor

[saminfatehi]

Hi @LifeofLucidity,

Hmm... That's strange. does your computer crash on step 2?
Reduce num_worker parameter to 1 (-num_worker=1). Maybe it will help.

[HeptanesianVibrionic]

Hi @saminfatehir,

Thanks for the quick response.

My computer does not crash on step 2, instead, it simply sits there indefinitely. For about 5 to 7 minutes the CPU and memory usage spikes before going back down with no changes in the terminal. I tried letting it run as long as possible to be certain processes weren't simply taking a long time.

Your suggestion of reducing the number of workers to 1 helped a certain amount. It seems to have progressed an additional few steps into the process before stagnating again without progress. This can be seen in my attached screenshot.

[saminfatehi]

Hi @LifeofLucidity,

It seems step 2 to have passed successfully and the bert server is ready to serve the requests. Now you can try step 3 in the other terminal.

[HeptanesianVibrionic]

Hi @saminfatehir,

I wasn't aware that I had to open a second terminal to complete step 3. I was under the impression I would wait for a process to finish so I could begin step 3 in the exact same terminal.

However, you are completely correct. After opening a second terminal and starting step 3, it ran to completion with no issues.

I ran into a hiccup when attempting to run predictor.py, which I have posted in the issue pertaining to that subject.

Many thanks for your engagement.