SDA: Survey Documentation and Analysis

• Home
• Archive
• Community
• Documentation
• Projects
• News

Projects

SDA Projects Under Development

• Stay tuned ...

Completed Projects

• SDA 4.1.4: This update includes several enhancements:
  • The C++ programs are 64-bit executables in SDA 4.1.4. (In previous versions of SDA they were 32-bit.)
  • MySQL 8 is now supported (along with the continuing support of MySQL 5.7)
  • The configuration for automatically sending emails about programmed nightly maintenance tasks is now done interactively via the SDAMANAGER. (Previously this configuration had to be done by editing an XML file on the server.)
  • Many other miscellaneous changes: documentation improvements, version upgrades to libraries, bug fixes, etc.

• SDA 4.1.3: This update includes several enhancements to XCODEBK's HTML output. SDA codebooks were improved stylistically, made easier to navigate and made fully compliant with the HTML 5 standard. The HTML output of the various analysis programs was also modified for HTML 5 compliance. However, the changes do not affect the appearance of the output in any significant way.

• SDA 4.1.2: This update enabled statistical tables created by TABLES and MEANS to be downloaded in CSV format for import into other applications (such as Excel).

• SDA 4.1.1: This update allows non-ASCII character variables in the raw input data file. The data file must be encoded in UTF-8, but character variables are no longer limited to the US-ASCII subset of characters.

• SDA 4.1: This update was focused on the SDA Manager
  

  The SDA Manager web app was enhanced in a number of ways:

  • SPSS .sav files, Stata .dta files, CSV files and TSV files can be imported and automatically converted into SDA datasets.
  • The user interface was reorganized to enhance usability.
  • The online help was extended and improved.
  • A CASEID variable is no longer required in an SDA dataset or in SDA metadata (DDL).

• SDA 4.0: servlet-based architecture, dataset-level access control, private workspaces and more ...
  

  SDA 4.0 converted completely from CGI to Java servlets for the web user interface. (The previous SDA interface was a blend of CGI and servlet-based components.) The SDA 4.0 enhancements included:

  • A user interface that's more like a desktop application with rich, dynamic components. At the same time, this allowed the simplification of the interface for beginning users by hiding more advanced options until they're needed.
  • For an archivist who is setting up an SDA archive, the installation procedure is simplified. The coordination of CGI and servlet-based components -- which can be tricky to configure -- will no longer be necessary.
  • A new SDA Manager web application is part of the SDA distribution package. The SDA Manager consolidates all of the management functions for an SDA archive into one web app. And all of the configuration information for an SDA archive is consolidated into one location -- a MySQL database. Previously, the management functions were handled by separate applications and the configuration information was dispersed in various files across the system.
  • Using the new SDA Manager web app, an archive manager can easily configure dataset-level access control -- specifying which users can access which datasets.
  • The SDA Manager web app also allows configuration of personal user workspaces. These user workspaces enable analysts to create and store recoded and computed variables in their own private storage areas.

  However, it's important to note that many of the pieces of the previous SDA architecture remained the same as before. In particular, the SDA assets that organizations have developed -- such as SDA datsets, codebooks, etc. -- are fully compatible with the new servlet-based web interface. The things that DIDN'T change in the new architecture include:

  • The SDA dataset format
  • The SDA codebook format
  • The SDA metadata syntax (DDL).
  • All the non-CGI SDA programs (analysis programs, codebook creation program, etc.)

  The things that DID change are:

  • The CGI programs (hsda, hsda2, etc.) were completely replaced by a servlet-based user interface.
  • SDA 4.0 requires ONLY a servlet container (Tomcat) to run. An additional web server -- Apache or IIS -- is no longer required to provide CGI capabilities.
  • The configuration information that was previously stored in HARC files is now stored in the MySQL database. (However, a simple procedure is provided to import the information from existing HARC files into the database.)

• Complex Standard Errors
  The calculation of standard errors for percentages and means in complex samples was revised to incorporate new developments for subpopulations and for differences. Complex standard errors for regression coefficients were implemented in phase 2 of this project. This was a joint project of CSM and the Substance Abuse and Mental Health Data Archive at ICPSR, University of Michigan.
• Charts
  Charts were added to the output of the TABLES program in version 1.4 and to the MEANS program in version 3.4. Charts were added to the SDA regression and logit programs SDA 4.0.3.
• Data Documentation Initiative (DDI)
  Many organizations are working together to arrive at a common standard for documenting data files using XML. CSM is participating in this ongoing effort and has been involved in developing tools to read and write DDI metadata files.
• Subset for Recoded and Computed Variables
  The subset procedure was revised in SDA 3.3, in order to allow recoded and computed variables to be included in a subset dataset. A new CSV (comma separated values) output format was added, including a header record with the names of variables, to facilitate reading the dataset into a spreadsheet.
• Disclosure Specifications to Protect Confidentiality
  The analysis programs were modified in SDA 3.3 to suppress output that may compromise the confidentiality of survey respondents. The analysis programs will all read a 'disclosure.txt' file (if one has been created for a study), and they will enforce the specifications included in that file.
• Search
  Search functionality was added to SDA 3.2. Variable-level search can be done within a single study. Both variable-level and study-level search can be done across studies.
• SDA Archiver
  The Archiver is a web application (released with SDA 3.1) that facilitates the work of adding new datasets and codebooks to an SDA archive. This application could be used both by administrators of a data archive and by individuals affiliated with member institutions who would like to set up their own dataset(s) in SDA.
• New User Interface
  A new user interface was developed for SDA 3.0 that includes program selection, variable selection, and program option choices all on the same screen. This was a joint project of CSM and ICPSR at the University of Michigan.
• Quick Tables
  Quick Tables is a simplified interface for obtaining analysis results. This was a joint project of CSM and ICPSR at the University of Michigan. For details about how it works, see the Quick Tables Information Page. You can also try generating a few Quick Tables for the GSS Cumulative Datafile from the SDA Archive page.
• "Word" Format Codebooks
  SDA 1.3 introduced a method of producing codebooks in Microsoft Word format. This new output option not only produces more attractive print codebooks -- with various fonts, heading styles, etc. -- but also allows users to easily introduce extra material, such as charts and other graphics, into print codebooks. Word-format codebooks are also a useful format because they can be automatically converted into PDF format using Adobe's Acrobat software. Here's an example of a small codebook produced by SDA 1.3 in Word format. And here's the same codebook after it's been automatically converted to PDF format (using Adobe Acrobat).
• Instrument Documentation (IDOC)
  CSM and the U.S. Census Bureau were involved in a joint project to construct web-browsable documentation for CAI instruments.

Last modified: Aug. 22, 2023 (Charlie Thomas, sda@berkeley.edu)