CopyCat Developer v. 3.9

CopyCat Developer 3.9 released!

Changes in this release include:
  • Improvements in the RTC inverted connection mechanism.
  • Major improvements in the trigger mechanism when in ReplicateOnlyChangedFields mode. The list of fields is now calculated dynamically in the trigger (instead of being hard-coded in the trigger), meaning that you don’t need to recreate the triggers after adding a field. It also means that the triggers create no dependency on your fields, so you can drop fields or even change the primary key definition without having to drop and recreate your triggers.
  • Fixed a bug causing replication to abort incorrectly after a timeout when a connection was lost due to a timeout from a very slow connection, when replicating over a transport layer (RTC or XMLRPC), sometimes resulting in a row being wrongly deleted from the destination database.
  • Various other minor bug fixes

CopyCat LiveMirror 1.5

LiveMirror 1.5 is available, bringing a number of small bug fixes, as well as a vastly improved trigger mechanism allowing to replicate changes field by field rather than by whole rows. This means that problems that used to occur with replication order (particalarly foreign key violations that were sometimes inevitable), should now never happen at all.

  The new trigger mechanism also allows for the primary key fields of the changed row to be found dynamically, meaning that you can now drop or recreate a primary key without having to recreate the triggers, and the change will automatically be detected in the triggers. This makes LiveMirror more seamless than ever.

  If you’ve been experiencing any problems with previous vesions of LiveMirror, please give this new release a try! And don’t forget you can contact us through the support forum if you need any help getting things working.

CopyCat Developer 3.8.2

Changes in this release:
  • New feature in RTC transport to allow backwards connections: the client contacts the server, the server keeps the client connection open until needed, and can therefore replicate towards the client when needed (upon initiative from the server). That means that you can put all the replication logic on the server rather than the client, even if the client has no static IP or is behind a firewall.
  • Added support for Delphi XE8 and Delphi 10 Seattle.
  • Added new option for tracking inconsistent deleted when using an RTC transport over a flaky unreliable connection. Property TCcReplicator.TrackInconsistentDeletes (disabled by default, and should only be used with RTC).
  • TCcReplicator: fixed an issue that sometimes occured during replication when ReplicateOnlyChangedFields=true (cf. http://copycat.fr/wordpress/discussions/#/discussion/87/issues-in-copycat-3-8-1-)
  • TCcReplicator: fixed an issue causing tracing mechanism to fail.
  • TCcConfig: ConnectAndRemoveConfig method didn’t remove RPL$LOG_VALUES
  • TCcReplicator: added caching of PK data types to avoid having to perform a query to find it out for every record during replication.
  • TCcReplicator: fixed a few bugs related to conflict management and resolution when using the new MergeChangedFields option.
  • Various minor bug fixes.

CopyCat Developer 3.8.1

New error handling mechanism:
  • Removed RPL$ERRORS table and TCcReplicator.LogErrors property.
  • Added some new fields to RPL$LOG (ERROR_MESSAGE, ERROR_CONTEXT).
  • Rows that cannot be replicated are now left in RPL$LOG with their ERROR_MESSAGE and ERROR_CONTEXT set.
New option to keep rows in RPL$LOG
  • A new property TCcReplicator.KeepRowsInLog allows you to keep replicated rows in RPL$LOG. This gives you a very detailed trace of everything that took place, and can be used instead of the depecated RPL$TRACE mechanism for debug purposes, or for auditing.
  • Rows that have been replicated will have their RPL$LOG.REPLICATION_STATE set, so if you want to see the rows that need to be replicated, you check where REPLICATION_STATE is null.
  • TCcReplicator: If a row can’t be replicated due to an error, we now place it in a list of failed rows, and continue to replicate. If the same row was changed again at a later point in the repliaction cycle, we skip it and leave it unreplicated. This is particularly important when using the new ReplicateOnlyChangedFields option, as data changes could otherwise get lost (and to avoid such problems, it would have been necessary to perform the entire replication cycle within a single transaction).
  • TCcReplictor: Added a new SkipToRemote option in OnReplicationError event allowing you to skip replication of local rows if an error occurs, but to continue replicating the remote changes to the local database. This feature makes sense only if using ReplicateOnlyChangedFields. Using this option, you can ensure consistency (by rolling back when there is a replication error) without aborting replication altogether.
  • TCcReplicator: Added queryPerformed parameter to OnRowReplFinishing and OnRowReplicated events, so that you can know what type of action was performed on the row (delete, update, or insert).
  • TCcConnection: fixed a bug that sometimes caused an access violation when destroying a TCcConnection.
  • Fix for SQL Server to correctly handle primary key fields of date or timestamp type.
  • TCcConfig: fixed a bug causing the OnExecute event of the TCcConnection not to fire while using TCcConfig.
  • TCcConfig: Added OnProgress event allowing you to keep the GUI active (using an Application.ProcessMessages for example) while configuring the databases.
  • TCcConfig: Some internal meta-data caching during configuration so as to improve performance over slow connections.
  • TCcConflictMgr: fixed a bug causing all the rows in RPL$LOG for a given row/table to be set to conflicted state rather than just the rows being replicated.
  • FIREBIRD ONLY: Fixed a bug causing the new ReplicateOnlyChangedFields feature to fail when replicating floating point values on systems using a decimal separator other than ‘.’.
  • FIREBIRD ONLY: Added a new RPL$NO_REPLICATION context variable. If you set this variable to ‘True’, the replication triggers will be by-passed. This is very useful if you want to execute a script that will cause massive updates to a large table.
  • TCcLog: fixed a bug that caused the conflict detection mechanism to fail sometimes.
  • TCcLog: fixed a bug that caused very bad performance when detecting a conflict using ReplicateOnlyChangedFields, in cases where there were a lot of changes in RPL$LOG.
  • TCcQuery: Fixed a bug causing the OnExecute event not to get fired.

CopyCat Developer v.3.7.1

We’re pleased to announce the release of CopyCat Developer version 3.7.1! The changes in this release are as follows: – XE7 support – Major performance and memory management improvements of big logs. Instead of loading the whole list of changes to replicate into memory at once, it is now streamed in row by row. – The UPDATE OR INSERT statement is now used in FB 2.1+, in order to reduce the number of statements needed and thus optimize performance. – Internal changes enabling the use of prepared queries during replication, thus again dramatically improving performance. – Improved performance of TCcConnection.GetFieldType – Fixed a memory leak that caused problems with long-running replication cycles. – Fixed a bug in the millisecond handling in Firebird.

Case study from Control-Bit Technologies

New case study from Control-Bit Technologies!
Control-Bit Technologies is a software manufacturing company that has specialized in the field of computerized security systems since 1985.

Our software has been installed in banks, military bases, high-tech facilities, courts and other places, with many years of successful operation, close personal service and support.  Our products use the most advanced software tools, meeting the customers’ needs and adapting to various kinds of hardware can be achieved quickly and inexpensively.

SESAM is an excellent cost-effective site management software system, which enables easy integration of diverse systems into a simple and intuitive control and reporting command center. SESAM works in Client/Server mode over networks, the intranet, and Internet. SESAM is ideal for industrial complexes, campuses, hotels, shopping centers, correctional, municipal, and other facilities.SESAM is a vendor-independent system that can be easily integrated into previously installed equipment from different vendors. New hardware systems are easily installed with no need to redefine existing applications.

Database and Challanges

We are using Firebird SQL, which is stable, quick, and free (since version 1.5 in 2007, now using version 2.5.2).

Our applications use multiple databases, for various purposes: System configuration database, automatic event logging, photo archiving, and more.

Part of our solutions is to run multi-node hot backups, and remote worker nodes, which requires semi real-time database replication.

A hot backup solution includes a primary server that replicated its databases to the secondary standby server.

Big projects include multiple hot backups, and more that 20 remote worker nodes. The master active server replicates data to all nodes, over LANs and WANs. Some installations are based on slow radio networks, a fact that adds more challenges to achieving a well functioning product which is based on databases.

CopyCat Implementation

We replicate multiple databases to multiple nodes. Some databases are being modified less frequently (system configuration), while others are being updated every second (history logging). Each database has its own record size, update frequency, and network connection speed. We are running multiple processes: one process for each master database. Each replication process has a separate thread per replication node, since some nodes may be off-line, and the network speed to each node is different.

Technical details

  • 10 GB — the size of the history database, old records are automatically cleared
  • 2,000,000 records in history database (limit is only because older history is not required)
  • Approximately 50,000 new records per day (depends on customer application)

Customer Support

During the development and integration of CopyCat into our products, we need technical support. We are impressed by the quick and professional support, including remote session into our computer.

New case study !

A new case study from one of our long-time customers !

 

Our Challenge

Our organization distributes Christian literature through volunteer groups based in two European countries. We needed a central database to keep a record of when, where, and how many pieces of literature were distributed by our multiple groups. This database also had to be accessible by individual members in a manner that was efficient and allowed for various levels of access.

The Solution

To manage the central database, we used MySQL 5.X, with a PHP interface that was developed by CopyCat ; this interface was essential for facilitating safe and encrypted access to the database. For client access, we used Firebird 2.X. Using this framework, and with the support of CopyCat, we were able to accomplish the following:
  1. The central database was accessible by each member of each group on their local computer, even when working offline.
  2. All new data was automatically compiled and updated within the central database.
  3. Each group’s access to the database was restricted, such that a group could change the administrative options for its own geographical region, but not the options for another group’s region.
  4. Within a given group, members who were higher in the administration were allowed greater levels of access to their system.

Conclusion

Given CopyCat’s software and developer support, we were able to design an easy to use, and easy to manage, central database with efficient and flexible client access.

Meet us at the Firebird conference in Prague

I will be speaking at the Firebird conference in Prague on the 24th and 25th of October 2014, presenting recent changes in CopyCat, offering a hands-on demonstation, answering questions, and so on… So if you are curious about CopyCat and plan to attend the conference, I hope to meet up with you !