v4.8

Overview:

In this release we introduce support for AWS S3 file storage, providing Dataverse installations with a cloud option. We also include support for Large Data upload via rsync and integration with an external application, the Data Capture Module (DCM). Other enhancements include improved Swift object storage, csv file ingest improvements, support for increased password complexity, downloading large guestbooks, removal of a user's roles, improved documentation, and various bug fixes.

• Provide S3 storage driver
• Improved Swift storage driver
• Support for large data upload and download workflows using rsync and DCM
• Improved CSV ingest.
• Configurable password complexity
• Fix and improve downloading of data for large guestbooks
• Fixed problem publishing a dataset with unescaped characters in title
• Disable user by button to remove all assigned roles
• Submit for review API endpoint
• Return to author API endpoint
• Create public-only configuration setting
• Restrict file API endpoint
• Improve migration documentation
• Improve User Guide section on permissions
• Provide installation-wide setting to control which metadata fields appear at top of dataset page

As always, thanks to all members of the Dataverse Community who contributed to this release by submitting suggestions, code, or other changes. Special thanks to Brian Silverstein, Oscar Smith, Rohit Bhattacharjee, and Sarah Ferry for their work on CSV and S3/Swift. Thanks to Pete Meyer for all the work on Rsync. Thanks to Don Sizemore and Akio Sone from Odum for fixes having to do with Glassfish and SPSS. Thanks to Solomon HM for improving migration documentation. Thanks to Jacob Makar-Limanov for work on UI accessibility.

For the complete list of issues, see the 4.8 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory, start
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
   • service glassfish start
3. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.8.war
4. Run the database update script.
   psql -U <db user> -d <db name> -f upgrade_v4.7.1_to_v4.8.sql
5. Run the workflow script.
   psql -U <db user> -d <db name> -f 3561-update.sql
6. Update citation.tsv to add grant agency and number to facet and advanced search.

curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"

Change in behavior note:
In this release, when a dataset is submitted for review by an author, the dataset no longer may be edited by the curator while it is in review. It must be returned to author before it can be modified.

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/forum/#!forum/dataverse-migration-wg

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.8/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f `find . -name '*.prep'`

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.7.1

Overview:

This release introduces a user management view to the administrator dashboard, listing relevant user information, providing search and super user promotion/demotion functionality. A few other community requested fixes and modifications are also provided.

• Display installation users in table format.
• Provide super user toggle functionality.
• Add additional information to user table: creation time, last login, last api use.
• Open search API to not require an API key.
• Properly validate username field on account creation to prevent non-functioning accounts.
• Add Department as a dataverse category.
• Prevent the Files facet on the MyData page hanging.
• On dataset page, autofill Identifier Scheme field with ORCiD id if available/logged in.

As always, thanks to all members of the Dataverse Community who contributed to this release by submitting suggestions, code, or other changes.

For the complete list of issues, see the 4.7.1 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory, start
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
   • service glassfish start
3. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.7.1.war
4. Run the database update script.
   psql -U -d -f upgrade_v4.7_to_v4.7.1.sql
   Note: this script will also remove any partially created, not functioning user accounts that may have resulted from a bug fixed in this release of not validating username on account creation.
5. Optionally restore old behavior of requiring API tokens to use Search API.
   -Search API does not require token now but if want to preserve old behavior run command:

curl -X PUT -d true http://localhost:8080/api/admin/settings/:SearchApiRequiresToken

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/forum/#!forum/dataverse-migration-wg

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.7.1/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f `find . -name '*.prep'`

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.7

Overview:

This release provides more customization and branding options for installations, improves documentation, provides better interoperability with citation tools, and incorporates code and bug fixes contributed by the Dataverse developer community during our recent community meeting hackathon.

• Allow creating a custom homepage, header, footer, navbar logo.
• Remove the system generated word Dataverse from dataverse names, making it optional.
• Make all system notifications use the name of the root dataverse in place of the word Dataverse.
• Make About link optional, off by default, on if link is specified.
• Make default guides link be an entry point to User Guide only to allow for other installation-specific structures.
• Allow specifying guides URL and overriding guides URL versioning mechanism.
• User and API Guide improvements contributed during the community meeting. Thank you, @acme146 for your help proofreading the User Guide!
• Developer Guide improvements, updates to README.md and CONTRIBUTING.md
• Add metadata tags to dataset page to improve interoperability with citations tools: Zotero, Endnote, Altmetrics
• Redirect after log in to page of origin for Shibboleth users. Thank you @aivanov100 for pull request #3910 and @donsizemore for testing it!

Big thanks to all members of the Dataverse Community who contributed to this release by submitting suggestions, code, or other changes.

For the complete list of issues, see the 4.7 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory, start
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
   • service glassfish start
3. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.7.war
4. Optionally run the database update script.
   Run this script if you want to preserve the word Dataverse after your current dataverse names. Uncomment the UPDATE line before running it.
   psql -U -d -f upgrade_v4.6.2_to_v4.7.sql
5. Optionally reindex all
   If you do not wish to keep Dataverse in existing dataverse names and so did not run the update script in step 4, you should reindex so that search cards reflect the new names.

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/forum/#!forum/dataverse-migration-wg

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.7/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.6.2

Overview:

This release includes improvements to mapping tabular data via WorldMap, support for object storage using Swift, support for Handles as persistent identifiers, improvements to the guides and various bug fixes:

• Fixed classification for latitude/longitude maps
• Restored WorldMap preview modal
• Added map thumbnail to dataset page
• Remove map data that subsequently becomes restricted
• Verify WorldMap links remain valid
• Improved Geoconnect documentation based on UX review
• Support for Handles as persistent identifiers
• Support for optional cloud-based object storage using Swift, including download URLs
• Support for alternative local identifier schemes, eg. sequential numbers rather than random strings
• Allow users to upload a dataset thumbnail
• Support for non-interactive installation
• A new Style Guide section for developers, Patterns
• Updated Two Ravens documentation
• Updated Developer Guide to reflect current process
• Fixed Guestbook Response Download to properly handle records for deleted files

Big thanks to all members of the Dataverse Community who contributed to this release by submitting suggestions, code, or other changes. Special thanks to the groups from DANS and CIMMYT that worked to restore the Handles functionality in the application.

For the complete list of issues, see the 4.6.2 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.

   • {glassfish install path}/glassfish4/bin/asadmin list-applications
   • {glassfish install path}/glassfish4/bin/asadmin undeploy dataverse

2. Stop glassfish and remove the generated directory, start

   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
   • service glassfish start

3. Deploy this version.

   • {glassfish install path}/glassfish4/bin/asadmin deploy {path}/dataverse-4.6.2.war

4. Run the database update script.
   psql -U {db user} -d {db name} -f upgrade_v4.6.1_to_v4.6.2.sql

5. Update the citations metadatablock:
   We have made a minor update to the Citations metadata configuration: The "Related Publication" fields
   will now be included on the "Create New Dataset" page. (And not just on the "Edit Metadata" page for
   existing datasets. To keep your Citation metadata configuration up-to-date, re-ingest the metadata
   block file supplied with the Dataverse distribution, as follows:

   curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @data/metadatablocks/citation.tsv -H "Content-type: text/tab-separated-values"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/forum/#!forum/dataverse-migration-wg

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.6.2/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.6.1

Overview:

This release introduces support for ORCID authentication, a file replace feature and several important enhancements:

• Support for OAuth2 remote authentication using ORCID, Google, or Github
• File replace functionality makes updating files easier
• A file version history on the file landing page
• A download URL for public files
• A reworked Log in page includes support for Remote Authentication Only mode
• Support uploading and replacing files with Native API
• Include file tags in metadata returned through the API
• Fixed a mapping bug introduced in v4.6 that prevents editing an existing map
• Fixed a request access bug affecting access to a single restricted file
• Some native API endpoint support CORS, allowing client-side Javascript code to use them (see here for native API guide - CORS enabled endpoints are labeled).

For the complete list of issues, see the 4.6.1 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.

   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse

2. Stop glassfish and remove the generated directory

   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated

3. If not yet present, add a jvm option to /usr/local/glassfish4/glassfish/domains/domain1/config/domain.xml to support timers, start glassfish

   • -Ddataverse.timerServer=true (note, see http://guides.dataverse.org/en/latest/admin/timers.html if using more than one server)
   • service glassfish start

4. Verify the two jhove files are in the config directory:
   For the file uploads to work properly, please follow the instructions in the README.txt file in https://github.com/IQSS/dataverse/tree/master/conf/jhove (In future version of the Dataverse the installer script will be taking care of this).

5. Deploy this version.

   • {glassfish install path}/glassfish4/bin/asadmin deploy {path}dataverse-4.6.1.war

6. Run the database update script.
   psql -U {db user} -d {db name} -f upgrade_v4.6_to_v4.6.1.sql

7. If using Shibboleth, update the configuration to work with the new Log In page
   Refer to the "Add the Shibboleth Authentication Provider to Dataverse" instructions on adding an authenticationProviders:
   http://guides.dataverse.org/en/4.6.1/installation/shibboleth.html

8. Remove old Shibboleth configuration setting :
   From the console of a server running Dataverse,
   curl -X DELETE http://localhost:8080/api/admin/settings/:ShibEnabled

9. You might want to make use of the new :DefaultAuthProvider setting if you want an auth provider other than Username/Email to be the default: http://guides.dataverse.org/en/4.6.1/installation/config.html#defaultauthprovider

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.6/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.6

Overview:

This release introduces a new File Landing Page and several important enhancements:

• Introduce a new File Landing Page
• Improve Deaccession Behavior
• API clean up
• OAI-PMH compliance improvements
• Optionally support SHA1 in place of MD5 checksum
• Fix an important issue with Request Access workflow
• Improve File Upload behavior, particularly drag and drop
• Document how to run Dataverse with SELinux enabled

For the complete list of issues, see the 4.6 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
3. If not yet present, add a jvm option to /usr/local/glassfish4/glassfish/domains/domain1/config/domain.xml to support timers, start glassfish
   • -Ddataverse.timerServer=true (note, see http://guides.dataverse.org/en/latest/admin/timers.html if using more than one server)
   • service glassfish start
4. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.6.war
5. Run the database update script.
   psql -U -d -f upgrade_v4.5.1_to_v4.6.sql
6. Update tool tip in metadata block.
   • Download the attached, latest social science metadata block file (social_science.tsv) to your glassfish server:
   • Update metadata block with latest .tsv file. From glassfish server in directory where .tsv was downloaded:
     • curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @./social_science.tsv -H "Content-type: text/tab-separated-values"
7. Update schema.xml.
   • Stop running solr process (kill -9 )
   • Replace current schema.xml file with latest attached version.
   • Restart solr (java -jar start.jar &)
8. Run incremental index.
   • Clear timestamps, from glassfish server run:
     • curl -X DELETE http://localhost:8080/api/admin/index/timestamps
   • Run incremental index, from glassfish server run:
     • curl http://localhost:8080/api/admin/index/continue

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.6/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.5.1

Overview:

The main feature of this release is the restoration of the IP Groups functionality. Additional features include email verification, dynamic copyright in the page footer, and support for Piwik.

• #700 IP groups in Dataverse are a way to specify a certain IP address range so that permissions can be applied based on the user's location or the device used to access the application. This is useful for installations that want to allow traffic from certain networks.
• #2170 Email verification encourages new users to confirm their account email address.
• #2905 Dynamic copyright automatically keeps the copyright date current and allows installations to add information about their organization.
• #3018 Piwik support provides an alternative to Google Analytics, that supports local collection and hosting of usage metrics.

Special thanks to Jeremy Richard from Sciences Po who contributed the Piwik support feature.

For the complete list of issues, see the 4.5.1 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
3. If not yet present, add a jvm option to /usr/local/glasssfish4/glassfish/domains/domain1/config/domain.xml to support timers, start glassfish
   • -Ddataverse.timerServer=true (note, see http://guides.dataverse.org/en/latest/admin/timers.html if using more than one server)
   • service glassfish start
4. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.5.1.war
5. Run the database update script.
   "psql -U -d -f upgrade_v4.5_to_4.5.1.sql"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.5.1/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.5

Overview:

The main features of this release are Harvesting, Metadata Export, Private URL, and Federated Login Mode.

• Harvesting allows the dataset metadata from another site to be imported so that these files appear to be local, though data files remain on the remote site. This allows Dataverse installations and other repositories to share metadata with each other to create a data sharing community and to provide more access to the datasets stored in each repository. Harvesting is implemented using the standard OAI-PMH protocol - any Dataverse installation can be configured as an OAI client and server.
• Metadata Export makes it easier to share and discover from Dataverse. Another key function of the export is to store the metadata in the file system in a format for preservation. In this release, the metadata is exported in Dublin Core, Data Documentation Initiative (DDI), and native JSON. Metadata export will support additional standards in the future.
• Private URL provides a simple way to generate and send a special URL to an unpublished dataset for a group of anonymous peer reviewers to review the data.
• Federated Login Mode allows users from 200+ different organizations to sign in using his or her organizational credentials.

Big thanks to all the community members who participated in usability testing, provided feedback in Github issues, and worked with us in other ways to make sure that this release was successful.

For the complete list of issues, see the 4.5 milestone in Github.

For help with upgrading, installing, or general questions please email [email protected].

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Stop glassfish and remove the generated directory
   • service glassfish stop
   • remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
3. Add a new jvm option to /usr/local/glasssfish4/glassfish/domains/domain1/config/domain.xml to support timers, start glassfish
   • -Ddataverse.timerServer=true (note, see http://guides.dataverse.org/en/latest/admin/timers.html if using more than one server)
   • service glassfish start
4. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.5.war
5. Run the database update script.
   "psql -U -d -f upgrade_v4.4_to_4.5.sql"
6. Replace the Solr schema.xml file
   • stop solr (kill -9 pid)
   • replace schema.xml
   • start solr (java -jar start.jar &)
7. Run the incremental index to prepare for harvesting facet.
   • Remove last index times from the database:
     update dvobject set indextime=null;
     update dvobject set permissionindextime=null;
   • From the glassfish console, run the incremental index:
     curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.5/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:

cd [files directory]
rm -f find . -name '*.prep'

If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.

v4.4

Overview:

This release contains feature enhancements for embedding content using widgets, downloading guestbook data for a dataverse in a consolidated fashion, and introduces a new feature, support for remote authentication using Shibboleth.

• Two new widgets: dataset and citation in addition to the existing widgets, dataverse listing and search box.
• Improved widget navigation: child objects open within widgets rather than redirecting to Dataverse.
• A new dataset metadata field, alternativeURL, can be used to link a dataset to an author's website.
• Shibboleth for remote authentication is now officially supported.
• Downloading guestbook data is now easier: all guestbook data for all datasets within a dataverse can be downloaded by one click.

Please see the <a href=https://github.com/IQSS/dataverse/issues?q=is%3Aissue+milestone%3A4.4+is%3Aclosed>4.4 milestone issues list for more information.

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse
2. Deploy this version.
   • /glassfish4/bin/asadmin deploy dataverse-4.4.war
3. Restart glassfish.
   • service glassfish stop
   • service glassfish start
4. Update the citation.tsv file
   • place the tsv file on the web server.
   • run the api command to update the citation metadata block:
     curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"
5. Replace the Solr schema.xml file
   • stop solr
   • replace schema.xml
   • start solr
6. Run the incremental index to correct the targeting for citation links.
   • Remove last index times from the database:
     update dvobject set indextime=null;
     update dvobject set permissionindextime=null;
   • From the glassfish console, run the incremental index:
     curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.

If you install Two Ravens or have an older version of Two Ravens configured and upgrade Two Ravens and Dataverse, you will need to update the preprocess.R file from the Two Ravens project:

1. Update preprocess.R
   replace the current preprocess.R file in:
   /usr/local/glassfish4/glassfish/domains/domain1/applications/dataverse-4.4/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R

with the one from:

https://raw.githubusercontent.com/IQSS/TwoRavens/master/rook/preprocess/preprocess.R

1. Remove any existing preprocess files

cd /usr/local/glassfish4/glassfish/domains/domain1/files
rm -f find . -name '*.prep'

Note the special apostrophe in the rm statement, it is not a single quote. Copy and pasting is the easiest way.

1. Restart glassfish

v4.3.1

Overview:

This release is a small patch release to address a potential security issue discovered by Andy Boughton. It also includes some unrelated minor changes.

• Escape script elements in HTML-enabled fields.
• Added support for downloading the citation in BibTeX format.
• Add Laboratory and Research Group as categories of dataverses.
• Correct a number of file tag issues, improving their function.

Please see the <a href=https://github.com/IQSS/dataverse/issues?q=is%3Aissue+milestone%3A4.4+is%3Aclosed>4.4 milestone issues list for more information.

Installation:

If this is a new installation, please see our Installation Guide.

Upgrade:

If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:

1. Undeploy the previous version.
   • /glassfish4/bin/asadmin list-applications
   • /glassfish4/bin/asadmin undeploy dataverse.war
2. Deploy this version.
   • /glassfish4/bin/asadmin deploy /dataverse-4.3.1.war
3. Restart glassfish.
   • service glassfish stop
   • service glassfish start
4. Replace the Solr schema.xml file
   • stop solr
   • replace schema.xml
   • start solr
5. Run the incremental index to correct the targeting for citation links.
   • Remove last index times from the database:
     update dvobject set indextime=null;
     update dvobject set permissionindextime=null;
   • From the glassfish console, run the incremental index:
     curl -X GET "http://localhost:8080/api/admin/index/continue?numPartitions=1&partitionIdToProcess=0"

If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/d/msgid/dataverse-migration-wg

Please note: v4.x does not currently support creating new handles though it will support existing ones. We intend to add this feature but have not yet scheduled this work.