ElassandraDatacenter resource not defined

[paulbarbu]

When I try to deploy a datacenter:

$ helm2 install --namespace "elassandra-logs" --name "elassandra-logs-cl1-dc1" --set replicas=3,elasticsearch.enabled=true,reaper.enabled=true --wait strapdata/elassandra-datacenter

Error: validation failed: unable to recognize "": no matches for kind "ElassandraDatacenter" in version "elassandra.strapdata
.com/v1beta1"                          

No custom configurations, just followed the quickstart guide, what should we do?

I reckon the CRD is missing?

[paulbarbu]

If I get the ElassandraDatacener (and Task) resources from https://github.com/strapdata/elassandra-operator/tree/f4105792e2a302210f2f829c27e9fefa642246e2/java/model/src/main/resources and then do
helm2 install --namespace "elassandra-logs" --name "logs-cl1-dc1" --set replicas=3,elasticsearch.enabled=true,reaper.enabled=true --wait strapdata/elassandra-datacenter

I get:

Error: release logs-cl1-dc1 failed: Internal error occurred: failed calling webhook "elassandradatacenter-webhook.elassandra.strapdata.com": Post https://elassandra-operator.elassandra.svc:443/validation?timeout=15s: x509: certificate is valid for elassandra-operator.default.svc, elassandra-operator.default.svc.cluster.local, not elassandra-operator.elassandra.svc

also helm2 ls --all logs-cl1-dc1 yields:

NAME            REVISION        UPDATED                         STATUS  CHART                           APP VERSION     NAME$
PACE                                                                                                                         
logs-cl1-dc1    1               Fri Sep 18 08:08:13 2020        FAILED  elassandra-datacenter-0.3.0     1.1             elassandra-logs

[paulbarbu]

Huh, so upon reading the error in more detail it seems there is a certificate issued only for the default namespace.
So this limits my options of installing elassandra operator in an other namespace.

[paulbarbu]

I installed the operator in the default namespace, but now, upon datacenter deployment I get yet another error:

$ helm2 install --namespace "elassandra-logs" --name "logs-cl1-dc1" --set replicas=3,elasticsearch.enabled=true,reaper.enabled=true --wait strapdata/elassandra-datacenter

Error: release logs-cl1-dc1 failed: Internal error occurred: failed calling webhook "elassandradatacenter-webhook.elassandra.strapdata.com": expected webhook response of admission.k8s.io/v1, Kind=AdmissionReview, got /, Kind=

[paulbarbu]

This is not going well.
I disabled operator's admission webhook (via helm ... -f values.yaml) and I seee yet another error in the operator's logs:

$ kubectl logs elassandra-operator-dfcff8cdc-hx26k

Listening for transport dt_socket at address: 5005
VM settings:
    Max. Heap Size: 512.00M
    Ergonomics Machine Class: server
    Using VM: OpenJDK 64-Bit Server VM

2020-09-18 09:24:55,004 INFO  [main] DefaultEnvironment.java:210 <init> Established active environments: [k8s, cloud]
2020-09-18 09:25:09,509 INFO  [pool-1-thread-2] TasksCleaner.java:52 onStartup Starting task cleaner period=138240000
2020-09-18 09:25:09,807 INFO  [main] JmxmpServerProvider.java:67 createJMXMPServer JMXMP started server=service:jmx:jmxmp://elassandra-operator-dfcff8cdc-hx26k:7199
2020-09-18 09:25:12,710 INFO  [main] Micronaut.java:89 lambda$start$2 Startup completed in 18792ms. Server Running: https://elassandra-operator-dfcff8cdc-hx26k:443
io.kubernetes.client.openapi.ApiException: okhttp3.internal.http2.ConnectionShutdownException
	at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:898)
	at io.kubernetes.client.openapi.apis.ApiextensionsV1beta1Api.createCustomResourceDefinitionWithHttpInfo(ApiextensionsV1beta1Api.java:178)
	at io.kubernetes.client.openapi.apis.ApiextensionsV1beta1Api.createCustomResourceDefinition(ApiextensionsV1beta1Api.java:153)
	at com.strapdata.strapkop.preflight.CreateCustomResourceDefinitions.createCrdFromResource(CreateCustomResourceDefinitions.java:63)
	at com.strapdata.strapkop.preflight.CreateCustomResourceDefinitions.call(CreateCustomResourceDefinitions.java:49)
	at com.strapdata.strapkop.preflight.CreateCustomResourceDefinitions.call(CreateCustomResourceDefinitions.java:37)
	at com.strapdata.strapkop.preflight.PreflightService.onStartup(PreflightService.java:57)
	at com.strapdata.strapkop.preflight.$PreflightServiceDefinition$Intercepted.$$access0(Unknown Source)
	at com.strapdata.strapkop.preflight.$PreflightServiceDefinition$Intercepted$$proxy0.invokeInternal(Unknown Source)
	at io.micronaut.context.AbstractExecutableMethod.invoke(AbstractExecutableMethod.java:146)
	at io.micronaut.aop.chain.MethodInterceptorChain.proceed(MethodInterceptorChain.java:60)
	at io.micronaut.scheduling.async.AsyncInterceptor.lambda$intercept$3(AsyncInterceptor.java:87)
	at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:79)
	at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:144)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: okhttp3.internal.http2.ConnectionShutdownException
	at okhttp3.internal.http2.Http2Connection.newStream(Http2Connection.java:242)
	at okhttp3.internal.http2.Http2Connection.newStream(Http2Connection.java:225)
	at okhttp3.internal.http2.Http2ExchangeCodec.writeRequestHeaders(Http2ExchangeCodec.java:116)
	at okhttp3.internal.connection.Exchange.writeRequestHeaders(Exchange.java:72)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:43)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at io.kubernetes.client.openapi.ApiClient$2.intercept(ApiClient.java:1232)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
	at okhttp3.RealCall.execute(RealCall.java:81)
	at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:894)
	... 20 more
2020-09-18 09:26:10,201 INFO  [elassandra-tasks-cleaner] TasksCleaner.java:86 run cleaner iteration fails due to : java.net.SocketException: Broken pipe (Write failed)
io.kubernetes.client.openapi.ApiException: java.net.SocketException: Broken pipe (Write failed)
	at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:898)
	at com.strapdata.strapkop.k8s.K8sResourceUtils$1TaskPage.<init>(K8sResourceUtils.java:887)
	at com.strapdata.strapkop.k8s.K8sResourceUtils.listNamespacedTask(K8sResourceUtils.java:907)
	at com.strapdata.strapkop.reconcilier.TasksCleaner$Cleaner.run(TasksCleaner.java:76)
	at java.util.TimerThread.mainLoop(Timer.java:555)
	at java.util.TimerThread.run(Timer.java:505)
Caused by: java.net.SocketException: Broken pipe (Write failed)
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
	at sun.security.ssl.OutputRecord.writeBuffer(OutputRecord.java:431)
	at sun.security.ssl.OutputRecord.write(OutputRecord.java:417)
	at sun.security.ssl.SSLSocketImpl.writeRecordInternal(SSLSocketImpl.java:894)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:865)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123)
	at okio.Okio$1.write(Okio.java:79)
	at okio.AsyncTimeout$1.write(AsyncTimeout.java:180)
	at okio.RealBufferedSink.flush(RealBufferedSink.java:224)
	at okhttp3.internal.http2.Http2Writer.settings(Http2Writer.java:185)
	at okhttp3.internal.http2.Http2Connection.start(Http2Connection.java:499)
	at okhttp3.internal.http2.Http2Connection.start(Http2Connection.java:489)
	at okhttp3.internal.connection.RealConnection.startHttp2(RealConnection.java:315)
	at okhttp3.internal.connection.RealConnection.establishProtocol(RealConnection.java:304)
	at okhttp3.internal.connection.RealConnection.connect(RealConnection.java:185)
	at okhttp3.internal.connection.ExchangeFinder.findConnection(ExchangeFinder.java:224)
	at okhttp3.internal.connection.ExchangeFinder.findHealthyConnection(ExchangeFinder.java:108)
	at okhttp3.internal.connection.ExchangeFinder.find(ExchangeFinder.java:88)
	at okhttp3.internal.connection.Transmitter.newExchange(Transmitter.java:169)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:41)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:221)
	at okhttp3.RealCall.execute(RealCall.java:81)
	at io.kubernetes.client.openapi.ApiClient.execute(ApiClient.java:894)
	... 5 common frames omitted