You can get it via NuGet for any .NET framework type.

Example

Let’s suppose we have a simple DB table User. Each user belongs to a UserGroup and owns several items, stored in UserItem. The tables are “connected” via common IDs and SQL foreign keys. The DB classes were generated via common tools like “dotnet ef scaffold”.

We want to get an overview of the users, including a property of the parent (UserGroup.Name) and a sum of the user’s items. The table is huge, so the whole preparation and calculation should be done by the server to reduce network traffic.

The data will be stored in a flat ViewModel class UserView, which contains the same structure as User, but with 2 additional fields, UserGroupName and UserItemSum.

First we prepare AutoMapper:

//using AutoMapper;
Mapper.Initialize(cfg =>
{
    cfg.CreateMap<User, UserView>()
      .ForMember(uview => uview.UserItemSum, opt => opt.MapFrom(user => user.UserItem.Sum(item => item.Number)))
});

Create the context instance for DB access:

var db = new mydbContext();

var dto = db.User.ProjectTo<UserView>().ToList();

//example for proceeding
myBindingSourceForDataGrid.DataSource = dto;

…and that was about it. We do not need any more Include() of subtables, Where(), Select() or anything else, and the whole query is run on the server side (the SQL server).

AutoMapper is smart enough to recognize that the UserView.UserGroupName column should belong to the user’s UserGroup.Name property.

In CreateMap(), we just need to tell how it handles the UserItemSum column.

Taking a look at the SQL query log, we see that the query in fact contains only the needed data:

SELECT `User`.`Fullname`, `User.UserGroup`.`Name` AS `UserGroupName`, `User`.`id`, (
    SELECT SUM(`item`.`Number`)
    FROM `mydb`.`UserItem` AS `item`
    WHERE `User`.`id` = `item`.`userId`
) AS `UserItemSum`
FROM `mydb`.`User` AS `User`
INNER JOIN `mydb`.`UserGroup` AS `User.UserGroup` ON `User`.`UserGroupid` = `User.UserGroup`.`id`

References

http://docs.automapper.org/en/stable/Queryable-Extensions.html

 [:]

Upgrading Ubuntu from v14 to 16 LTS can lead to MySQL server issues (MySQL is upgraded from v5.6 to 5.7 during the upgrade process).

E.g. if you use deprecated variables in /etc/mysql/my.cnf, MySQL server will fail to start after the upgrade is finished.

My recommendations for the upgrade:

1. If the server is a virtual appliance, create a snapshot when the system is shut down. Otherwise create a full backup.
2. Within Ubuntu v14, do an “apt-get update && apt-get upgrade” and check if everything is running fine.
3. Create a snapshot again.
4. Edit /etc/mysql/my.cnf. In my case I had to delete variables
   1. table_cache
   2. log-slow-queries
   3. (long query time)
5. Run “do-release-upgrade” to v16 Xenial.
6. If you are using phpMyAdmin, the dbcommon upgrade procedure might fail during the upgrade process (“cannot access mysqld.sock” or similar). Simply ignore it for now.
7. When the upgrade is finished, check if MySQL is running (“ps ax”) and if not, try to start it manually (e.g. “service mysql restart”).
   1. If MySQL server is unable to start, look at /var/log/mysql/error.log, maybe you have to modify my.cnf again, e.g. remove further certain variables. In this case, I recommend to note the vars and start from the latest snapshot again.
8. If you have huge MySQL databases, watch /var/log/mysql/error.log and/or mysql.log. It might take several minutes as some DB types might be converted to newer formats in all tables.
9. Check if phpMyAdmin works. If Dbcommon failed during the upgrade, run “dpkg-reconfigure phpmyadmin”.

Additions:

It turned out that more modifications are necessary:

MySQL 5.7 seems to ignore max_connections value. If it is e.g. set to 600 in my.cnf, it still uses value 214.

To solve this issue, we have to do a few more adaptions.

LimitNOFILE=infinity
LimitMEMLOCK=infinity

* soft nofile 1024000
* hard nofile 1024000
* soft nproc 10240
* hard nproc 10240

open_files_limit = 1024000
table_open_cache = 500

Optionally, also decrease the timeout values:

wait_timeout = 5000
interactive_timeout = 5000

Run afterwards:

systemctl daemon-reload
service mysql restart

Check the currently used max_connections value e.g. with phpMyAdmin or MySQL console.

Update 2017-05-04:

If you encounter further MySQL errors like “ERROR 1366 Incorrect integer value: ” for column ‘xx’ at row xx, you might want to change sql_mode’s value to the default one from v5.6 (if modifying the SQL query is not an option):

sql_mode=NO_ENGINE_SUBSTITUTION

Reference

Reference[:]

As there is no way to set a “friendly name” value in the DB table structure directly, you might think of putting some additional data into the .Net database entity  model. This is not always a good idea, because data could get lost when you update your model structure from DB.

So the most useful way I found so far in ASP.Net is to use a combination of System.ComponentModel.DataAnnotations and Model Metadata.

Example

You have a table mytable in your model mymodel, which contains a column mycolumn. The friendly name for mycolumn should be “My Column”.

In your Models folder (Visual Studio Solution Explorer), create 2 class files called MetaData.cs and PartialClasses.cs.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.ComponentModel.DataAnnotations;

namespace myproject.Models
{
    public class MetaData
    {
        public class mytableMetaData
        {
            [Display(Name = "My Column")]
            public object mycolumn;
            

        }
    }
}

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.ComponentModel.DataAnnotations;

namespace myproject.Models
{    
    [MetadataType(typeof(MetaData.mytableMetaData))]
    public partial class mytable
    {
    }
}

In your .cshtml file, use:

@Html.DisplayNameFor(m => m.mycolumn)

You should now see the friendly column name “My Column” in your browser.

E.g. “@Html.ValidationSummary()” now also uses the display name from Metadata.

If you refresh your model structure later, these two classes are not affected.

Reference[:]

Execute query:

select concat('KILL ',id,';') from information_schema.processlist where user='yourUser'

Copy the result lines and execute them, it should look like:

kill 12345;
kill 12346;
kill 12347;
...

Reference[:]

mysql -u root -p

Run:

SET global log_output = 'table';
SET global general_log = 1;

View table mysql.general_log e.g. with phpMyAdmin or command:

select * from mysql.general_log

Do not forget to disable logging again:

SET global general_log = 0;

Source

An SQL query containing multiple table joins, functions etc. is too slow, but necessary.

Solution approaches

Indexes created on single or multiple MySQL table columns can increase performance in a significant way if they are used correctly.

First, have a look at the MySQL Explain command. Simply add “explain ” at the beginning of your query and execute it. E.g. in phpMyAdmin, you get a table which shows which keys (indexes) are available and which ones are used. If you see many NULL values or high row/reference numbers in this table, your table indexes have to get optimized.

If you have a query like this,

select bla from `user` where lastname = 'asdf'

it is a good idea to create an index for column “lastname”.

Using this query,

select max(logindate) from `user` where lastname = 'asdf'

the index should contain both columns “logindate” and “lastname”. Note, if you create an index “(lastname, logindate)”, a separate index containing only “lastname” is not needed any more.

Using multiple joins like,

select a.asdf, b.bsdf, c.csdf
from a
left join a.id on b.aid
left join a.id on c.aid

make sure each of b.aid and c.aid is indexed. Additionally, the joined columns must have the same type and length for the index to be used.

Reference

You have to move a MySQL database table to another database, but you have many other tables, functions, routines, events etc. which access it. Before moving it, it is not enough to find only the tables containing foreign keys, but also find functions etc. which use it.

This PHP script does that for you: mysql_table_references

Usage

Edit the script to enter your DB server login data, then run it in your browser and enter the table name which you want to find references for.

Script code

<!DOCTYPE html>
<html>
<head>
 <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">

</head>
<body>
<?php
    $searchtable = @$_GET['searchtable'];
?>

<h1>References, Views and Routines regarding Table</h1>

<form action="<?=$_SERVER['PHP_SELF']?>" method="get">
Table name: <input type="text" name="searchtable" value="<?=@$searchtable?>">
<input type="submit" value="Search">
</form>

<?php
    if (!$searchtable)
        exit;

?>

<?php

$dbhost = 'localhost';
$dbname = 'yourdb';
$dbuser = 'root';
$dbpw = 'yourpw';

try
{
    $db = new PDO("mysql:host=$dbhost;", $dbuser, $dbpw);
}
catch(PDOException $ex)
{
    echo "<b>Could not connect to server! Please make sure you have set the correct config values in this file.</b>";
    echo "<br>Details:<br>".$ex->getMessage();
    exit;
}

//Foreign Keys:
$result = $db->query("select
  TABLE_NAME,COLUMN_NAME,CONSTRAINT_NAME, TABLE_SCHEMA, REFERENCED_TABLE_NAME,REFERENCED_COLUMN_NAME
from INFORMATION_SCHEMA.KEY_COLUMN_USAGE
where
  REFERENCED_TABLE_NAME = '$searchtable' or TABLE_NAME = '$searchtable'
  order by TABLE_NAME, REFERENCED_TABLE_NAME;");
//Alternative?
//  show create table $searchtable;
?>

<div>Note: Results may contain unrelated entries if the table name is not unique, e.g. if it is also used in column names etc.</div>

<hr>
<h2>Results for <b><?=$searchtable?></b></h2>

<h3>Foreign Keys</h3>
<table border="1">
<tr>
    <th>DB</th>
    <th>Table</th>
    <th>Column</th>
    <th>Referenced Table</th>
    <th>Referenced Column</th>
</tr>

<?php
while($row = $result->fetch())
{
     ?>
     <tr>
        <td><?=$row['TABLE_SCHEMA']?></td>
        <td><?=$row['TABLE_NAME']?></td>
        <td><?=$row['COLUMN_NAME']?></td>
        <td><?=$row['REFERENCED_TABLE_NAME']?></td>
        <td><?=$row['REFERENCED_COLUMN_NAME']?></td>
     </tr>
     <?php
}
?>
</table>
<?php

//Usage in Views:
$result = $db -> query("select
    TABLE_SCHEMA, TABLE_NAME, VIEW_DEFINITION
    from INFORMATION_SCHEMA.views
    where view_definition like '%from%$searchtable%'
");

?>
<h3>Views</h3>
<table border="1">
<tr>
    <th>DB</th>
    <th>Table</th>
    <th>View Definition</th>
</tr>

<?php

while($row = $result->fetch())
{
     ?>
     <tr>
        <td><?=$row['TABLE_SCHEMA']?></td>
        <td><b><?=$row['TABLE_NAME']?></b></td>
        <td><small><?=nl2br(str_ireplace($searchtable, "<b>$searchtable</b>", $row['VIEW_DEFINITION']))?></small></td>
     </tr>
     <?php
}

?>
</table>
<?php

//Usage in Routines:
$result = $db -> query("select
    ROUTINE_SCHEMA, ROUTINE_NAME, ROUTINE_DEFINITION
    from INFORMATION_SCHEMA.ROUTINES
    where routine_definition like '%from%$searchtable%'
");
?>
<h3>Routines</h3>
<table border="1">
<tr>
    <th>DB</th>
    <th>Table</th>
    <th>Routine Definition</th>
</tr>

<?php

while($row = $result->fetch())
{
     ?>
     <tr>
        <td><?=$row['ROUTINE_SCHEMA']?></td>
        <td><b><?=$row['ROUTINE_NAME']?></b></td>
        <td><small><?=nl2br(str_ireplace($searchtable, "<b>$searchtable</b>", $row['ROUTINE_DEFINITION']))?></small></td>

     </tr>
     <?php
}

?>
</table>
<?php

//Usage in Events:
$result = $db -> query("select
    EVENT_SCHEMA, EVENT_NAME, EVENT_DEFINITION
    from INFORMATION_SCHEMA.events
    where event_definition like '%from%$searchtable%'
");

?>
<h3>Events</h3>
<table border="1">
<tr>
    <th>DB</th>
    <th>Table</th>
    <th>Event Definition</th>
</tr>

<?php

while($row = $result->fetch())
{
     ?>
     <tr>
        <td><?=$row['EVENT_SCHEMA']?></td>
        <td><b><?=$row['EVENT_NAME']?></b></td>
        <td><small><?=nl2br(str_ireplace($searchtable, "<b>$searchtable</b>", $row['EVENT_DEFINITION']))?></small></td>

     </tr>
     <?php
}

?>
</table>

<script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.2.0/js/bootstrap.min.js"></script>
</body>
</html>

It might also be useful to get all DB users which have access to the table(s). Use phpMyAdmin for this (Activate table, then click “Permissions”).

Moving tables to another database might have effect to a lot of software accessing the data, and it could take a long time to get the code updated. During this process, use a workaround like table views. You can reference the new table with the view, so it is accessible via both databases. Using MySQL, you can e.g. use “update” or “insert” commands directly on the view in many cases, e.g. if there is exactly 1 table in the view query.

Example: create view address as select * from newdb.address

Framework CakePHP sets strings to null which are loaded from database tables and contain special characters like German Umlaute.

Solution

In app/Config/database.php, add to your config array: ‘encoding’ => ‘utf8’

Example:

public $default = array(
		'datasource' => 'Database/Mysql',
		'persistent' => false,
		'host' => 'localhost',
		'login' => 'asdf',
		'password' => 'meow',
		'database' => 'bsdf',
		'prefix' => '',
		'encoding' => 'utf8',
	);

It seems like this works even if your DB columns have another charset / collation like latin1_swedish_ci.

Notes

If you are using Ajax, make sure you clean the browser’s cache after modifying the .php file. Otherwise the changes might not be shown immediately.

Basically, it would be also a good idea to use UTF-8 in your DB and all your .php / .ctp files.

• In MySQL, you have to change the default collation per DB, per table and also the columns themselves. phpMyAdmin is a good choice for doing this. Usually, no problem should appear if you e.g. change from latin to UTF-8, but to be safe create a backup first.
• Use collation utf8_unicode_ci, NOT utf8_general_ci! It could end in wrong results.
• For your files, if you use a PHP IDE like NuSphere PhpED: Right near the bar under the code window where the line numbers etc. are shown, you see the charset of the file. Click on it to change it to UTF-8. Windows Notepad has similar functions in the Open/Save dialogs.
• Make sure you set the correct charset in the header of your PHP files. In CakePHP, UTF-8 is already set in app/Config.core.php.

MySqlCommand cmd = new MySqlCommand("insert into common_history(datetime, username, text) values (@datetime, @username, @text)", adapter.UpdateCommand.Connection);
cmd.Parameters.Add(DateTime.Now);
cmd.Parameters.Add(Environment.UserName);
cmd.Parameters.Add(text);
cmd.BeginExecuteNonQuery();
cmd.Dispose();

Instead, use

MySqlCommand cmd = new MySqlCommand("insert into common_history(datetime, username, text) values (@datetime, @username, @text)", adapter.UpdateCommand.Connection);
cmd.Parameters.AddWithValue("@datetime", DateTime.Now);
cmd.Parameters.AddWithValue("@username",Environment.UserName);
cmd.Parameters.AddWithValue("@text", text);
cmd.BeginExecuteNonQuery();
cmd.Dispose();

zip_file_and_delete.bat contains:

%PROGRAMFILES%7-Zip7z a -t7z %1 %2
if errorlevel 1 goto :error
del %2
:error

Install 7-Zip to use compression.