dchart database dataset limit


Hello!

I'm having some trouble when adding datasets to my database, when i do a
> dapperload.sh ls anclajes_db
a list of all my datasets do appear, but when i access thru the dods, only a minor list appear, and so in the dchart interface. So instead of having my 600 datasets in the interface, it only shows like 150.

There is a way to turn this limit up?

(All my datasets are correct, and pointing to a netcdf file)

Regards.

Favio Medrano
Cicese

A couple of questions...


Hi Favio,

Did the output from dapperload.sh display indicate any errors (like "Invalid depth/height axis" or "Skipping profile ..."?

If not, do you have publicly available URL for your server?

- Joe

The logs


The error that i'm getting from dapper is this:

------------------------------------------------------------------------
Caused by: java.io.IOException: "java.io.IOException:Can't find dataset:PER_T3500_LR75DW_NS11357_Z715_CNK18_00_time"
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.initDods(NetcdfDataset.java:440)
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.init(NetcdfDataset.java:447)
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.(NetcdfDataset.java:308)
at gov.noaa.cdp.util.DatasetFactory.getDatasetContainer(DatasetFactory.java:94)
at gov.noaa.cdp.util.DatasetFactory.factory(DatasetFactory.java:109)
... 38 more

17:01:02,473 INFO DclientPlotStationDataset:83 - java.io.IOException: "java.io.IOException:Can't find dataset:PER_T3500_LR75DW_NS11357_Z715_CNK18_00_time"
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.initDods(NetcdfDataset.java:440)
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.init(NetcdfDataset.java:447)
at gov.noaa.cdp.util.NetcdfDataset$NetcdfProtoDataset.(NetcdfDataset.java:308)
at gov.noaa.cdp.util.DatasetFactory.getDatasetContainer(DatasetFactory.java:94)
at gov.noaa.cdp.util.DatasetFactory.factory(DatasetFactory.java:109)
at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at gov.noaa.pmel.dclient.data.DclientPlotStationDataset.getFactoryEntry(DclientPlotStationDataset.java:83)
at gov.noaa.pmel.dclient.data.DclientPlotStationDataset.init(DclientPlotStationDataset.java:254)
at gov.noaa.pmel.dclient.data.DclientPlotStationDataset.init(DclientPlotStationDataset.java:232)
at gov.noaa.pmel.dclient.data.DclientPlotStationDataset.(DclientPlotStationDataset.java:105)
at gov.noaa.pmel.dclient.data.DclientPlotDatasetFactory.createSoftReference(DclientPlotDatasetFactory.java:159)
at gov.noaa.pmel.dclient.data.DclientPlotDatasetFactory.addEntry(DclientPlotDatasetFactory.java:109)
at gov.noaa.pmel.dclient.data.DclientPlotDatasetFactory.init(DclientPlotDatasetFactory.java:95)
at gov.noaa.pmel.dclient.data.DclientPlotDatasetFactory.(DclientPlotDatasetFactory.java:66)
at gov.noaa.pmel.dclient.PlotHandler.getDatasetFactory(PlotHandler.java:150)
at gov.noaa.pmel.dclient.PlotHandler.genStationmarks(PlotHandler.java:551)
at gov.noaa.pmel.dclient.PlotHandler.handle(PlotHandler.java:226)
at gov.noaa.pmel.dclient.DclientServlet.doGet(DclientServlet.java:63)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at gov.noaa.pmel.dclient.ConfigFilter.doFilter(ConfigFilter.java:73)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at gov.noaa.pmel.dclient.CacheFilter.doFilter(CacheFilter.java:72)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:619)

----------------------------------------------------------------------

And this errors raises for all my other datasets that doesnt appear on dchart,
but when i do the following command for list the dataset, it gives it corrrectly

[root@cic-pem dchart]# ./dapperload.sh ls anclajes_db PER_T3500_LR75DW_NS11357_Z715_CNK18_00_time
file:/datos/NETCDF_DATA/ANCLAJES/CNK18-R/PER-T3500-CNK18/ADCPS/PER-T3500-LR75DW-NS11357-Z715-CNK18-20080920-20090719-00.nc
[root@cic-pem dchart]#

Any work around for this?

Regards
Favio Medrano

Check your dchart config file


A couple of other possibilities:

1) What URL are you using in your DChart configuration file? Does it end with ".cdp"?
2) Can you view the OPeNDAP data in your browser? For instance, if you go to:

http://<your host:8080>/dods/anclajes_db/

do you see a listing of your datasets?

found the error!


Hi! thank you for the reply.

Yes I found what was causing the error, it was one netcdf file that didn't contain any variable, the dapperload command was skipping this dataset. The output of this was: Couldn't find any variable in dataset skipping profile XXX

I removed that dataset, and that fixed the issue.
So yeah you where right thank you, I had to check the log of this commands, took long because I have about 1000 datasets.

Now my issue is different, I want to make the panel of the datasets wider, I made it already changing the css style, but the dataset container keeps the same size, which is managed by the "option_container" class.

But I can't find the source for this. Can you help me where can I change this, mostly because the names of my datasets are pretty long, and so the categories.

Or even change to make it resizeable?

Thank you for the great help!

Favio Medrano
CICESE

CSS for dataset container


You have to change the .box width parameter in dapper/dchart/html/style/dchart.css to change the dataset container width. Also, you need to change the .column2 left position to move the variables container to the right after you expand the dataset width. Finally, you'll need run ant in the ./dapper directory to copy the new css file to the Tomcat webapp directories.

- Joe

Hi Joe, thanks for the


Hi Joe, thanks for the reply

I already did that, I modified the dchart.css, and changed the width in .box, also changed the html so I have only 1 column.

But the control that contains the categories and datasets, remains the same size, it doesn't care the size of the box, maybe this constrained size needs to be changed from the source.

Can it be changed?

- Favio

Ok found it


Just to let you know that I found where to change the size of the datasets table. In js/dchart.js file.
The function function DchartDataTable(xmlDoc, props) contains the specifications for size of the table and also the sizes of each column.
That's why it didn't matter if I changed the dchart.css.

Regards
- Favio

Thanks...


Thanks for letting me know, and sorry about the confusion ... it's been awhile since I worked on that part of the JavaScript code and I'd forgotten that it's hard coded.

- Joe

No, no errors what so ever in


No, no errors what so ever in the dapperload.sh command.

In the dchart interface i notice that only shows 136 datasets, no matter from what database I choose, it seems like a limit for the interfaze.

my mysql server does not throw any error.

Some workaround i could be checking on?

Ps. sorry the server it still not open for public

Thanks!
-Favio Medrano