pb running NER_Keras.py code with Keras 1.0.3

[gpcoursera]

Dear Nils,
First of all thanks a lot for the great tutorial. I'm learning a lot watching your videos and reading through your code.
I naively wanted to run our implementation of the SENNA model with Keras and cannot make it work even after trying many things and spending of lot of time on forums dedicated to Theano and Keras.
For the record, I think it has something to do with The FixEmbedding layer and/or my version of Keras==1.0.3. I'm running Python 3.5 but I fixed all print and unicode trivial issues in the code so it should not be a pb with compatibility with Python 3.

Here is the traceback:

Traceback (most recent call last):
File "NER_Keras.py", line 126, in
model.add(FixedEmbedding(output_dim=embeddings.shape[1], input_dim=embeddings.shape[0], input_length=n_in, weights=[embeddings]))
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/models.py", line 114, in add
layer.create_input_layer(batch_input_shape, input_dtype)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 341, in create_input_layer
self(x)
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/keras/engine/topology.py", line 458, in call
self.build(input_shapes[0])
TypeError: build() takes 1 positional argument but 2 were given

Thanks for your help.
G.

[beinborn]

Dear G.,
I had the same problem. The code runs well with Keras 0.3, but not with newer versions. I used this instead of the FixedEmbedding:

embedding = Embedding(output_dim=embeddings.shape[1], input_dim=embeddings.shape[0], input_length=numInputUnits, weights=[embeddings])
embedding.trainable=[]
model.add(embedding)

Maybe this works for you too.
Best,
Lisa

[flekova]

Hi, if I remember well, the problem was caused by the SharedX import, which causes trouble in newer versions. You can comment that import out and instead define these in the FixedEmbeddings.py:

def shared_zeros(shape, dtype=theano.config.floatX, name='', n=1): shape = shape if n == 1 else (n,) + shape return theano.shared(np.zeros(shape, dtype=dtype), name=name) def sharedX(X, dtype=theano.config.floatX, name=None): return theano.shared(np.asarray(X, dtype=dtype), name=name) def floatX(val): return np.asarray(val, dtype=theano.config.floatX)

Then it shall work also in newer versions.

Cheers,
Lucie

[gpcoursera]

@beinborn : thanks Lisa. I followed your post and it works fine.
I was searching for this simple add-on: embedding.trainable=[]
to ensure I'm not retraining my embeddings at each step.
Thank you
G.

[gpcoursera]

But I don't get the performance announced by Nils (F-score around 0.7). Here are my benchmark:
10 epoch: F1 on dev: 0.665523, F1 on test: 0.668567

[stone8oy]

Thanks great tutorial. The NER_Keras.py implements the word-level method. How to implement the sentence-level log likelihood method for NER ? Can anybody give some hints ?

[nreimers]

@stone8oy Currently Keras is not supporting this. You would need to implement a HMM model by yourself and add it to Keras.

However, there is a CRF model for Keras which can be found in the keras contrib. repository. This is quite similar to the STC model for Keras.

In the near future (within this month), I will also publish some code on Github that includes a CRF model for Keras.