Deleting batch of rows from MS-SQL Server table
Deleting large number of rows from a (SQL Server) table can cause replication delay, create a very large transaction and a big performance impact, as well as escalating locks so that the table will be unavailable.
If it is possible use truncate:
TRUNCATE TABLE MyTable; |
Truncate will run much faster although it has no facility to filter rows, it does a table meta data change at the back (e.g. TRUNCATE will reset the IDENTITY value for the table if there is one).
The option of deleting small portions at a time will be slower, although it will generate less impact on the server performance.
DECLARE @COUNT INT SET @COUNT = 0 WHILE (@COUNT < 1000) BEGIN DELETE TOP (10000) FROM [MyTable] -- WHERE []; --SELECT CURRENT_TIMESTAMP; WAITFOR DELAY '00:00:02'; SET @COUNT = (@COUNT + 1) END |
Hibernate: How to insert OneToMany children by cascade
I needed a way to insert an entity with a new joined (children) entity together. One way to do it is by separate the actions (e.g. to save the joined entity first and then to save the main entity).
However, Hibernate support inserting a new entity with new joined entity.
I would like to present a way to insert new parent element together with its new OneToMany children by cascade:
The parent Hibernate entity:
@Entity @Table(name = "Parent") @Cache(usage = CacheConcurrencyStrategy.READ_WRITE) public class Parent { ... @Override @Id @GeneratedValue(strategy = GenerationType.AUTO) @Column( name="Parent_ID", unique = true, nullable = false, updatable = false ) public Integer getId() { return id; } private Long id; @OneToMany(fetch = FetchType.LAZY, cascade = { CascadeType.ALL,CascadeType.PERSIST,CascadeType.MERGE }, mappedBy = "parent") @Column(nullable = false) public List<Children> getChildrens() { return childrens; } public void setChildrens( List<Children> childrens) { this.childrens = childrens; } private List<Childrens> childrens; } |
The children Hibernate entity:
@Entity @Table(name = "Children") @Cache(usage = CacheConcurrencyStrategy.READ_WRITE) public class Children { ... @ManyToOne( fetch = FetchType.EAGER ) @JoinColumn(name = "parent_id", nullable = false ) public Parent getParent() { return parent; } public void setTitle(Title title) { this.parent = parent; } private Parent parent; } |
In the Java code:
Don’t forget to set the parent in the children entity.
For example:
Parent parent; … // getting the Hibernate entity of the parent ... List<Children> childrenList = new ArrayList<Children>(); Children children; children = new Children(); children.set... children.setParent( parent ); childrenList.add(children); title.setChildrens(childrenList); |
Solr spellcheck compound from several fields
The Solr’s SpellCheck component is designed to provide inline spell checking of queries (i.e. query suggestions or “Did You Mean”) in case it thinks the input query might have been misspelled. The words can be loaded from text files, a field in Solr, or even from several fields (more than one field).
In order for making the spellcheck loading the words from several fields you need to:
Declare a new field and copy all the fields, of which their words should be part of the spellcheck index, into the new field
The declaration of the new field and the copy should be configured in the the schema.xml file.
New field declaration
You should pay attension to the following properties:
- Field type: It is important that you declared it as field type textSpell (Solr’s Spellcheck only works with field type textSpell).
- MultiValued: because your index consist from several fields (not one) you must declare it as multi valued.
- Stored: for space issue, declare the field as not stored (in my case it was a difference between increase of 13% vs 43% of space).
<!-- multiple field spell check --> <field name="didYouMean" type="textSpell" indexed="true" stored="false" multiValued="true"/> |
Copy the fields into the new field
Suppose the following are the fields you would like your spell check consist from:
<field name="q" type="text" indexed="true" stored="true" /> <field name="tn" type="text" indexed="true" stored="true" /> <field name="an" type="text" indexed="true" stored="true" /> |
The following statements copy all the fields, of which their words should be part of the spellcheck index, into the new field.
<copyField source="q" dest="didYouMean"/> <copyField source="tn" dest="didYouMean"/> <copyField source="an" dest="didYouMean"/> |
Configure Solr to use the new field
The configuration of Solr to use the new field (to specify the field name on which the spell check will operate on) is done in the solrconfig.xml file.
<searchComponent name="spellcheck" class="solr.SpellCheckComponent"> <str name="queryAnalyzerFieldType">textSpell</str> <lst name="spellchecker"> <str name="name">default</str> <str name="field">didYouMean</str> <str name="spellcheckIndexDir">./spellchecker</str> <str name="buildOnCommit">true</str> </lst> </searchComponent> <requestHandler name="/spell" class="solr.SearchHandler" lazy="true"> <lst name="defaults"> <str name="spellcheck.onlyMorePopular">false</str> <str name="spellcheck.extendedResults">false</str> <str name="spellcheck.count">5</str> <str name="spellcheck">on</str> <str name="spellcheck.collate">true</str> </lst> <arr name="last-components"> <str>spellcheck</str> </arr> </requestHandler> |
How to make configuration changes into effect
In order to make the spellcheck configurations into effect follow the following steps:
- Restart your server (e.g. sudo /etc/init.d/jetty restart)
- Reload config and full import with cleaning (http://localhost:8983/solr/test/admin/dataimport.jsp?handler=/dataimport)
- Test your regular index (http://localhost:8983/solr/test/select/?q=*:*&start=0&rows=10&indent=on)
- Test the spell check index (http://localhost:8983/solr/test/spell/?q=helllo&version=2.2&start=0&rows=10&indent=on&spellcheck=true&spellcheck.build=true&spellcheck.collate=true)
Notes
- If you query for a sentence that contain more than one word, the spellcheck response will contain alternatives for each word not found in the index. The spellcheck.collate=true causes a modified version of the original query (the sentence) to be returned with the most likely alternatives.
- Note the spellcheck.build=true which is needed only once to build the spellcheck index from the main Solr index. It takes time and should not be specified with each request. SpellCheckComponent can be configured to automatically (re)build indices based on fields in Solr index when a commit is done. In order to do so you must enable this feature by adding the following line in your SpellCheckComponent configuration for each spellchecker where you wish it to apply:
<str name="buildOnCommit">true</str>
I will be happy to receive any comment from you.
How to create a copy of existing Solr index
If you have an already existing Solr index and you want to try new settings, it is best to work and test on a copy Solr’s index first.
Here are the simple steps to make a copy of your Solr’s index:
Suppose we have already an index named cast and the copy will be named castSpell.
- Make new directory under solr (e.g. castSpell).
- Copy the entire directories conf and lib from cast to castSpell.
- Make new directory called data under the new directory castspell.
- Add to solr.xml your new index name: <core instanceDir=”castSpell” name=”castSpell” />
- Restart Jetty (e.g. sudo /etc/init.d/jetty restart)
- Go to http://localhost:8983/solr/castSpell/admin/dataimport.jsp?handler=/dataimport
- click on Reload-config
- click on Full-import
- continue to next step only after the status became “idle” (by clicking on Status).
- Test your new index by: http://localhost:8983/solr/castSpell/select/?q=*:*&start=0&rows=10&indent=on
Drupal Sitemap does not show URL aliases
The Sitemap module
The Drupal Sitemap module generates a sitemap page and gives visitors a site overview. All categories and terms (i.e. vocabulary) are expanded optionally with node counts and RSS feeds.
The taxonomy URL paths on the sitemap page are not in the URL paths’ alias format
In Drupal 6, I encountered a problem: The taxonomy URL paths on the sitemap page are not in the URL paths’ alias format (which are generated by the Pathauto module). For example, the taxonomy URLs are in the format /taxonomy/term/28, instead of being displayed with the assigned URL alias, such as /category/projects.
Some of the solutions I found on the web are:
- Uninstalling and reinstalling the Sitemap and Pathauto modules;
- “Bulk update” in the Pathauto module settings;
- Changing modules weights.
However, none of the solutions listed above, nor any other solutions, worked for me. In order to fix this problem, I had to fix the Drupal Sitemap module code:
Code fix
Go to your sitemap module directory, back up and edit:
cd sites/all/modules/site_map cp site_map.module site_map.module.backup vi site_map.module |
At function _site_map_taxonomy_tree, inside the elseif ($term->count) block, replace this line:
$term_item .= l($term->name, ($cat_depth < 0) ? taxonomy_term_path($term) : "taxonomy/term/$term->tid/$cat_depth", array('attributes' => array('title' => $term->description))); |
with the following four lines:
/* show URL alias at sitemap */ $temp_term_path = taxonomy_term_path($term); $temp_path_alias = drupal_get_path_alias($temp_term_path); $term_item .= l($term->name, $temp_path_alias, array('attributes' => array('title' => $term->description))); |
This fixed the sitemap page to show the URL aliases (clean URL) in the taxonomy links (see http://www.quizmeup.com/sitemap).
If you have any comment or a suggestion, please share.
How to log all MySQL queries in Drupal
In order to benchmark a Drupal site performance you need to see all the database queries related to your Drupal site.
In case you don’t have access to the my.cnf file
If you don’t have access to the my.cnf file, you can log the queries from the Drupal code itself:
cp includes/database.mysql-common.inc includes/database.mysql-common.inc.backup.orig vim includes/database.mysql-common.inc |
At function db_query, just before the return clause, write:
error_log("Query: $query"); |
This will write all the queries Drupal uses for calculating its pages.
Be careful and use it with wisdom, as it might blow your error_log file.
To stop the queries logging process, overwrite the includes/database.mysql-common.inc file with the original.
cp includes/database.mysql-common.inc.backup.orig includes/database.mysql-common.inc |
In case you have access to the my.cnf file
If you have access to the my.cnf file, you need to set the log parameter (at the my.cnf file):
log = /var/log/mysql/mysql-queries.log |
The output will be at
/var/log/mysql/mysql-queries.log |
Please remember to shut the log off after you have finished, or it could hurt your site performance.
See MySQL queries log analyzing
Insert into multiple MySQL tables with one-to-one relationship
I have been asked if there is a way to insert data into two MyISAM tables that connected by a one-to-one relationship in one MySQL query.
For example:
mysql> CREATE TABLE `user` ( `id` int(10) unsigned NOT NULL auto_increment, `name` varchar(255) NOT NULL default '', PRIMARY KEY (`id`) ) ENGINE=MyISAM AUTO_INCREMENT=5 DEFAULT CHARSET=latin1 mysql> CREATE TABLE `user_details` ( `id` int(10) unsigned NOT NULL, `weight` int(10) unsigned default NULL, `height` int(10) unsigned default NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 |
How can I insert at once a new product with its details?
My answer is that you can’t insert into multiple MySQL tables in one query (if you know a way, I will be happy to learn;).
However, there are several simple and easy ways to do it:
Using select in the insert value:
INSERT INTO user (name) VALUES ('John Smith'); INSERT INTO user_details (id, weight, height) VALUES (SELECT(id FROM user WHERE name='John Smith'), 83, 185); |
Using LAST_INSERT_ID in the insert value:
INSERT INTO user (name) VALUES ('John Smith'); INSERT INTO user_details (id, weight, height) VALUES (LAST_INSERT_ID(),83, 185); |
Note: The value of mysql_insert_id() is affected only by statements issued within the current client connection. It is not affected by statements issued by other clients.
Using PHP script:
<?php // Connecting to database $link = mysql_connect($wgScriptsDBServerIP, $wgScriptsDBServerUsername, $wgScriptsDBServerPassword, true); if(!$link || !@mysql_SELECT_db($wgScriptsDBName, $link)) { echo("Cant connect to server"); exit; } // Values to insert $name = 'John Smith'; $weight = 83; $height = 185; // insertion to user table $sql = "INSERT INTO user (name) VALUES ('$name')"; $result = mysql_query( $sql,$conn ); // retrieve last id $user_id = mysql_insert_id( $conn ); mysql_free_result( $result ); // insertion to user_details table $sql = "INSERT INTO user_details (id, weight, height) VALUES ($user_id, $weight, $height)"; $result = mysql_query( $sql,$conn ); mysql_free_result( $result ); ?> |
If you know of another good way please post a comment.
Note: This post was fixed according to the comments I have received from Singer Wang, Shlomi Noach and Chris Evers. Thanks guys!
Connection Pool: MySQL Communications link failure
The Problem And The Solution
While using a MySQL connection pool in Java, I received a MySQL Communications link failure Exception (see below).
In order to solve communication link failure exception:
- I have removed JDBC property autoReconnect=true and put only the JDBC property autoReconnectForPools=true
- I have added the connection properties:
- testOnBorrow
- testWhileIdle
- timeBetweenEvictionRunsMillis
- minEvictableIdleTimeMillis
See for example: How to set up a MySQL connection pool in Java
The MySQL CommunicationsException: Communications link failure Exception
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error : ** BEGIN NESTED EXCEPTION ** com.mysql.jdbc.exceptions.jdbc4.CommunicationsException MESSAGE: Communications link failure Last packet sent to the server was 0 ms ago. STACKTRACE: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure Last packet sent to the server was 0 ms ago. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2985) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2871) at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3414) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1936) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536) at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1564) at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1485) ... at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost. at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2431) at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2882) ... 14 more ** END NESTED EXCEPTION ** |
Connection Pool: MySQL connection closed exception
The Problem
While using a MySQL connection pool in Java, I received a MySQL Connection Closed Exception (see below).
The problem was that the JDBC driver was not compatible to the MySQL server version. I have used version 5.05a for both JDBC and MySQL and it solved the problem
The problem is that the connection stayed idle longer than the MySQL wait_timeout (MySQL Server System Variables).
A (Simple) Solution:
1) Set the property minIdle to 0 (this is the default). That way no connection can stay idle for more than DEFAULT_MIN_EVICTABLE_IDLE_TIME_MILLIS (the minimum amount of time an object may sit idle in the pool).
2) Set the JDBC properties: autoReconnect=true and autoReconnectForPool=true
See for example: How to set up a MySQL connection pool in Java
The MySQL Connection Closed Exception
ERROR ... com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: No operations allowed after connection closed.Connection was implicitly closed due to underlying exception/error: ** BEGIN NESTED EXCEPTION ** com.mysql.jdbc.exceptions.jdbc4.CommunicationsException MESSAGE: The last packet successfully received from the server was36797 seconds ago. The last packet sent successfully to the server was 36797 seconds ago, which is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem. STACKTRACE: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was36797 seconds ago.The last packet sent successfully to the server was 36797 seconds ago, which is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1074) at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3246) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1917) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2060) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2536) at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1564) at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1485) ... Caused by: java.net.SocketException: Broken pipe at java.net.SocketOutputStream.socketWrite0(Native Method) at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) at java.net.SocketOutputStream.write(SocketOutputStream.java:136) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123) at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3227) ... 12 more ** END NESTED EXCEPTION ** |
How To Switch Java in Centos
* If you need to install a new version of java, download the version from java.sun
* run /usr/sbin/alternatives to change the default java. If you will run the next command you will see which java versions are available:
# /usr/sbin/alternatives --config java There is 1 programs which provide 'java'. Selection Command ----------------------------------------------- *+ 1 /usr/lib/jvm/jre-1.4.2-gcj/bin/java Enter to keep the current selection[+], or type selection number: |
Add the new java to the list of alternatives:
# /usr/sbin/alternatives --install /usr/bin/java java /usr/java/jdk1.6.0_15/bin/java 2 |
Choose the new java as the default:
# /usr/sbin/alternatives --config java There are 2 programs which provide 'java'. Selection Command ----------------------------------------------- *+ 1 /usr/lib/jvm/jre-1.4.2-gcj/bin/java 2 /usr/java/jdk1.6.0_15/bin/java Enter to keep the current selection[+], or type selection number: 2 |
The same thing you should do with for javac:
# /usr/sbin/alternatives --install /usr/bin/javac javac /usr/java/jdk1.6.0_15/bin/javac 2 # /usr/sbin/alternatives --config javac There are 2 programs which provide 'javac'. Selection Command ----------------------------------------------- *+ 1 /usr/lib/jvm/java-1.4.2-gcj/bin/javac 2 /usr/java/jdk1.6.0_15/bin/javac Enter to keep the current selection[+], or type selection number: 2 |
Adding DbUnit to your project
About DbUnit
DbUnit is a JUnit extension (also usable with Ant) targeted at database-driven projects that, among other things, puts your database into a known state between test runs. This is an excellent way to avoid the myriad of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage.
See http://www.dbunit.org/
How to add DbUnit to your project
Step 1: You need to install MySQL on your development machine
Configure it to have the following properties:
Port = 3306 User = test Password = test Schema = test |
Step 2: Add DbUnit dependency to your pom.xml
File: pom.xml:
<dependency> <groupId>org.dbunit</groupId> <artifactId>dbunit</artifactId> <version>2.4.7</version> <scope>test</scope> </dependency> |
Step 3: Create abstract class to extend DBTestCase
File: DbTest.java
package com.xxx.quality; import java.io.FileInputStream; import java.sql.Connection; import java.sql.DriverManager; import java.sql.Statement; import org.apache.commons.pool.ObjectPool; import org.apache.commons.pool.impl.GenericObjectPoolFactory; import org.dbunit.DBTestCase; import org.dbunit.PropertiesBasedJdbcDatabaseTester; import org.dbunit.dataset.IDataSet; import org.dbunit.dataset.xml.FlatXmlDataSetBuilder; public abstract class DbTest extends DBTestCase { private static final String DEFAULT_HOST = "localhost"; private static final int DEFAULT_PORT = 3306; private static final String DEFAULT_SCHEMA = "test"; private static final String DEFAULT_USER = "test"; private static final String DEFAULT_PASWSWORD = "test"; private ObjectPool pool; public DbTest() { System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_DRIVER_CLASS, "com.mysql.jdbc.Driver"); System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_CONNECTION_URL, "jdbc:mysql://" + DEFAULT_HOST + ":" + DEFAULT_PORT + "/" + DEFAULT_SCHEMA + "?autoReconnect=true"); System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_USERNAME, DEFAULT_USER); System.setProperty(PropertiesBasedJdbcDatabaseTester.DBUNIT_PASSWORD, DEFAULT_PASWSWORD); MySqlPoolableObjectFactory mySqlPoolableObjectFactory = new MySqlPoolableObjectFactory(DEFAULT_HOST, DEFAULT_PORT, DEFAULT_SCHEMA, DEFAULT_USER, DEFAULT_PASWSWORD); GenericObjectPoolFactory genericObjectPoolFactory = new GenericObjectPoolFactory(mySqlPoolableObjectFactory); pool = genericObjectPoolFactory.createPool(); } @Override protected IDataSet getDataSet() throws Exception { return new FlatXmlDataSetBuilder().build(new FileInputStream("src/test/java/com/answers/quality/cleandataset.xml")); } public ObjectPool connectionPool() { return pool; } } |
Step 4: create clean dataset
File: cleandataset.xml
<dataset> <MY_TABLE /> </dataset> |
Step 5: create the result dataset
File: dataset.xml
<dataset> <MY_TABLE field_name1 = "1" field_name2 = "41" /> <MY_TABLE field_name1 = "2" field_name2 = "71" /> </dataset> |
Step 6: Write your test
File: MyFirstTest.java
package com.answers.quality; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.sql.SQLException; import java.util.List; import org.dbunit.Assertion; import org.dbunit.DatabaseUnitException; import org.dbunit.database.IDatabaseConnection; import org.dbunit.dataset.DataSetException; import org.dbunit.dataset.IDataSet; import org.dbunit.dataset.ITable; import org.dbunit.dataset.SortedTable; import org.dbunit.dataset.xml.FlatXmlDataSet; import org.dbunit.dataset.xml.FlatXmlDataSetBuilder; import org.dbunit.operation.DatabaseOperation; import org.junit.Test; import com.google.common.collect.ImmutableList; public class MyFirstTest extends DbTest { public static final String INITIAL_STATE = "src/test/java/com/xxx/dataset.xml"; @Test public void testGetAllRecords() throws Exception { List<MyFirstDataObject> expectedRecords = ImmutableList.of( new MyFirstDataObject(...), new MyFirstDataObject(...) ); FlatXmlDataSet dataSet = new FlatXmlDataSetBuilder().build(new FileInputStream(INITIAL_STATE)); IDatabaseConnection connection = getConnection(); DatabaseOperation.CLEAN_INSERT.execute(connection, dataSet); MyFirst myFirst = new MyFirst(connectionPool()); int id = 3; List<MyFirstDataObject> actualRecords = MyFirst.getRecords(id); assertEquals(expectedRecords, actualRecords); } @Test public void testInsertRecord() throws Exception { // Load expected data from an XML dataset IDataSet expectedDataSet = new FlatXmlDataSetBuilder().build(new FileInputStream(INITIAL_STATE)); SortedTable expectedTable = getSortedTable(expectedDataSet); // Execute the tested code that modify the database here ... // Fetch database data after executing your code IDataSet databaseDataSet = getConnection().createDataSet(); SortedTable actualTable = getSortedTable(databaseDataSet); Assertion.assertEquals(actualTable, expectedTable); } private SortedTable getSortedTable(IDataSet databaseDataSet) throws DataSetException { ITable actualTable = databaseDataSet.getTable("MY_TABLE"); SortedTable sortedTable1 = new SortedTable(actualTable, new String[]{"field_name1", "field_name2"}); sortedTable1.setUseComparable(true); return sortedTable1; } } |
Remark:
In order to clean the test database every time you start the test, you can add a static block in DbTest class and put the cleaning MySQL statements there.
For example:
static{ try { Class.forName("com.mysql.jdbc.Driver").newInstance(); String url = "jdbc:mysql://" + DEFAULT_HOST + ":" + DEFAULT_PORT + "/" + DEFAULT_SCHEMA + "?autoReconnect=true"; Connection connection = DriverManager.getConnection(url, DEFAULT_USER, DEFAULT_PASWSWORD); Statement st = connection.createStatement(); st.execute("CREATE TABLE IF NOT EXISTS ...;"); st.close(); connection.close(); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } } |
How to get directories via FTP
How to GET directories via FTP:
You can get a directory files (recursive) by using wget command with flag -m for –mirror
wget -m ftp://username:password@ip.of.old.host |
See using wget to recursively download whole ftp directories