================= CCIT Telecon 2010-01-28 ================= Present: Bruce Wilson, Mike Daigle, Ryan Scherle, Roger Dahl, Matt Jones, Mark Servilla, Robert Sandusky, Robert Waltz, Paul Allen, Giri Palanisamy Topics to cover: * Progress on cn_service Robert: Discussion with Matt later today to discuss details of plan. Going forward with developing an implementation based on unit tests being written right now. Working on the XML validation. Work is based on Maven. ?? What are the requirements for build? Need Spring IDE, palm should provide all of the needed dependancies inside Eclipse. Matt: Need a command-line build for nightly build checks. ??What do we need to install to make this happen?? Robert to show how to do this (&&&&&ticket). Matt: REST services that Serhan wrote last summer, maps well to the CN services. May look at how to leverage Serhan's work to implement desired capability. https://trac.dataone.org/attachment/ticket/188/CNRestServletDesignv03.txt Matt: CN Deployment. Now an automated system for deployment to a CN Dev node. Makefile build Debian packages, which specify dependancies. Makefile builds each of these, puts in place, runs post-scripts for initiation. Need to only make changes to the system through the makefile and use make install for propagating changes. Giri: Worked with Matt on this for Mercury elements. Should be able to deploy Mercury using this. ?? What's the plan for getting the Mercury portion of the CN implemented? Giri to work with Jim and Robert. * Progress on mn_service Roger: Working on general refinements. Implemented put functionality for updating system metadata from CN. Finishing that should finish up the MN REST spec compliance. Will then start work on the instructions for how to deploy (plug into Apache). ?? MN and CN functions are very similar, such as for Get. But MN is being done in Python, but CN is being done in Java. Makes sharing difficult. CN and MN share a call for get(metadata). get(data) is a MN function. ??? How to handle large bit streams, multi-part bit streams, requests for subsets of a dataset. (future functionality). * Project metrics that can be collected from cyberinfrastructure * Log file content * Heartbeat methods * System monitor (Nagios) instruments and recording * Notes started at: http://mule1.dataone.org/ArchitectureDocs/logging.html Matt: Metacat keeps log of CRUD operations inside the database for all data & metadata operations. Covers different information than the Apache logs. M ay need to look beyond the Apache information for overall logging. Need to resolve the privacy issues around the log file information. Owner should be able to see at least some log information for the objets s/he owns. &&&& document as decision/ticket. Noted that 0.3 release has no authentication, so that makes some security issues deferred by design. Need to ensure that prvacy by design is appropriately considered. * Discuss [http://mule1.dataone.org/ArchitectureDocs/querying_content.html Query support] for CN.search(), MN.listRecords() and getLogRecords() of CN and MN. Probably needs more discussion, Matt made some changes. Do we want to support things like XQuery and other paths, which are not in teh currend design document. May want to provide better documentation for reasons for decisions. * Any outstanding logistics for upcoming meeting Agenda for meeting has been updated (today). Some further detail added, but still a skeletal agenda. &&&& Bruce to follow up with Dave, ensure we have a more fleshed out agenda. Matt to contact Carl to get him to prep for a discussion on Data Conservancy approach and be prepared for Day 2 discussion of leverage points between the two DataNEts for CI perspective. Suggestion, meet in Lobby of hotel at 6:00. * Update on activities of EVA group (Matt) Matt has been participating with group. Group is meeting in Ft. Collins this week. Building a set of analyses. Matt Demo'd Kepler and how this relates to niche modeling. Excited about VisTrails and how this relates to their work. Not so focussed on data access and data discoverry. They are focussed on analysis and assuming that the data access and data discovery will happen. Need to ensure that we allocate a sufficient percentage of effort to makings ure that they EVA have the tools that are needed within the investigator toolkit. EVA focussing on how to pull small bits of data out of things like the MODIS data, how to get pits together. This is an issue from the data access perspective -- see note above about subsetting. Bruce (notes on ESG and BER call). * Aligning the level of status updates with needs of stakeholders. Ryan: Volume of email out of trac and other places. Bruce: SAw about 130 e-mails over the past 3 days. Hard to separate the information from the chaff in the Trac spam. Who needs to see what Trac updates? Perhaps a role for more written information in e-mails? What about tracking/storage/searching of e-mails? Could we do more with information (updates) in e-mails, versus notes in meeting? This week's notes are somewhat detailed, and previous notes have migrated. Note: trac errors (including source: protocol) is a knwon issue with Trac to svn integration. Work is in an active ticket but will get resolved. Note can sue cc field to send any ticket to any given group. Currently using the mechanism so taht Trac spam goes to a particular group. Concern about whether or not people are missing something important if it's not called out in a particular e-mail, particularly for those who are less directly involved with what's doing on in the project. Something like a monthly or weekly synopsis of CCIT activities might be useful. Will help of monthly or weekly synopsis are in a consistent and findable location. Move to two levels -- those that get everything and those that get a synopsis and get directed at tickets that are specific for them. ?? Matt -- investigate if Trac will send tickets to people if they're assigned a ticket or on the CC list even if they're not on the master list? * Other issues Giri: requested Jim (Green) attend Thursday meetings. Matt and Giri need to coordinate how the Apache configuration can be modified to ensure that the configuration coesxists for Metacat and Mercury. Bruce: ORNL server has arrived. Minor shipping damage, but has not affected function. Will install Ubuntu server as a base OS, with KVM, and then can build coodinating node VM's for testing. Need to resolve some issues with networking charges.